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Abstract 

Model checking and testing are two areas with a similar goal: to verify that a system satisfies a 

property. They start with different hypothesis on the systems and develop many techniques with different 

^^ notions of approximation, when an exact verification may be computationally too hard. We present some 

p^j notions of approximation with their logic and statistics backgrounds, which yield several techniques 

, for model checking and testing: Bounded Model Checking, Approximate Model Checking, Approximate 

O Black-Box Checking, Approximate Model-based Testing and Approximate Probabilistic Model Checking. 

All these methods guarantee some quality and efficiency of the verification. 
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1 Introduction 

Model checking and Model-based testing are two methods for detecting faults in systems. Although similar 
in aims, these two approaches deal with very different entities. In model checking, a transition system (the 
model), which describes the system, is given and checked against some required or forbidden property. In 
testing, the executable system, called the Implementation Under Test (lUT) is given as a black box: one 
can only observe the behavior of the lUT on any chosen input, and then decide whether it is acceptable or 
not with respect to some description of its intended behavior. 

However, in both cases the notions of models and properties play key roles: in model checking, the goal 
is to decide if a transition system satisfies or not some given property, often given in a temporal logic, by 
an automatic procedure that explores the model according to the property; in model-based testing, the 
description of the intended behavior is often given as a transition system, and the goal is to verify that the 
lUT conforms to this description. Since the lUT is a black box, the verification process consists in using the 
description model to construct a sequence of tests, such that if the lUT passes them, then it conforms to 
the description. This is done under the assumption that the lUT behaves as some unknown, maybe infinite, 
transition system. 

An intermediate activity, black box checking combines model checking and testing as illustrated in the 
Figure fl] below, originally set up in |PVY99[ IYan04) . In this approach, the goal is to verify a property of a 
system, given as a black box. 

We concentrate on general results on efficient methods which guarantee some approximation, using basic 
techniques from complexity theory, as some tradeoff between feasibility and weakened objectives is needed. 
For example, in model checking some abstractions are made on the transition system according to the 
property to be checked. In testing, some assumptions are made on the lUT, like an upper bound on the 
number of states, or the uniformity of behavior on some input domain. These assumptions express the 
gap between the success of a finite test campaign and conformance. These abstractions or assumptions are 
specific to a given situation and generally do not fully guarantee the correctness. 

Model 
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Property [ P j " ► lUT 

Black Box Checking 



Implementation Under Test 

Figure 1: Model checking, black box checking and testing. 

This paper presents different notions of approximation which may be used in the context of model 
checking and testing. Current methods such as bounded model checking and abstraction, and most testing 
methods use some notions of approximation but it is difficult to quantify their quality. In this framework, 
hard problems for some complexity measure may become easier when both randomization and approximation 
are used. Randomization alone, i.e. algorithms of the class BPP may not suffice to obtain efficient solutions, 
as BPP may be equal to P. Approximate randomized algorithms trade approximation with efficiency, i.e. 
relax the correctness property in order to develop efficient methods which guarantee the quality of the 
approximation. This paper emphasizes the variety of possible approximations which may lead to efficient 
verification methods, in time polynomial or logarithmic in the size of the domain, or constant (independent 
of the size of the domain) , and the connections between some of them. 

Section 2 sets the framework for model checking and model-based testing. Section 3 introduces two 
kinds of approximations: approximate techniques for satisfiability, equivalence and counting problems, and 



randomized techniques for the approximate versions of satisfiabihty and equivalence problems. Abstraction 
as a method to approximate a model checking problem, Uniform generation and Counting, and Learning 
are introduced in section 3.1. Property testing, the basic approach to approximate decision and equivalence 
problems, as well as statistical learning are defined in Section 3.2. Section 4 describes the five different types 
of approximation that we review in this paper, based on the logic and statistics tools of Section 3 for model 
checking and testing: 

1. Bounded Model Checking where the computation paths are bounded (Section 4.1), 

2. Approximate Model Checking where we use two distinct approximations: the proportion of inputs which 
separate the model and the property, and some edit distance between a model and a property (Section 
4.2), 

3. Approximate Black Box Checking where one approximately learns a model (Section 4.3), 

4. Approximate Model-based Testing where one finds tests which approximately satisfy some coverage 
criterium (Section 4.4), 

5. Approximate Probabilistic Model Checking where one approximates the probabilities of satisfying for- 
mulas (Section 4.5). 

The methods we describe guarantee some quality of approximation and a complexity which ranges from 
polynomial in the size of the model, polynomial in the size of the representation of the model, to constant 
time: 

1. In bounded model checking, some upper bounds on the execution paths to witness some error are 
stated for some class of formulas. The method is polynomial in the size of the model. 

2. In approximate model checking, the methods guarantee with high probability that we discover some 
errors. We use two criteria. In the first approach, if the density of errors is larger than e, Monte Carlo 
methods find them with high probabilities in polynomial time. In the second approach, if the distance 
of the inputs to the property is larger than e, an error will be found with high probability. The time 
complexity is constant, i.e. independent of the size of the model but dependent on s. 

3. In approximate black box checking, learning techniques construct a model which can be compared with 
a property. Some intermediate steps, such as model checking are exponential in the size of the model. 
These steps can be approximated using the previous approximate model checking and guarantee that 
the model is e-close to the lUT after N samples, using learning techniques which depend on s. 

4. In approximate model-based testing, a coverage criterium is satisfied with high probability which 
depends on the number of tests. The method is polynomial in the size of the representation. 

5. In approximate probabilistic model checking, the estimated probabilities of satisfying formulas are close 
to the real ones. The method is polynomial in the size of a succinct representation. 

The paper focuses on approximate and randomized algorithms in model checking and model-based testing. 
Some common techniques and methods are pointed out. Not surprisingly the use of model checking techniques 
for model-based test generation has been extensively studied. Although of primary interest, this subject is 
not treated in this paper. 

We believe that this survey will encourage some cross-fertilization and new tools both for approximate 
and probabilistic model checking, and for randomized model-based testing. 



2 Classical methods in model checking and testing 

Let P be a finite set of atomic propositions, and 'P{P) the power set of P. A Transition System, or a 
Kripke structure, is a structure M. = (5, sq, -R, L) where 5 is a finite set of states, sq G S' is the initial state, 
i? C 5* X 5 is the transition relation between states and L : S" — >■ 'P{P) is the labelling function. This 
function assigns labels to states such that if p e P is an atomic proposition, then M., s\= p, i.e. s satisfies p 
if p e L{s). Unless otherwise stated, the size of M is 15*1, the size of S. 

A Labelled Transition System on a finite alphabet / is a structure C — {S, sq, ^, R^ L) where S, sq, L are 
as before and R C S x I x S. The transitions have labels in /. A run on a word w e /* is a sequence of 
states sq, Si, ...., s„ such that {si,Wi, Si+i) G P for i = 0, ..., n — 1. 

A Finite State Machine (FSM) is a structure T = {S, Sq, 1,0, R) with input alphabet / and output 
alphabet O and R(- S x I xO x S. An output word t e O* is produced by an input word w e /* of the FSM 
if there is a run, also called a trace, on w, i.e. a sequence of states so,Si, ...,Sn such that {si,Wi,ti, Si+i) € P 
for i = 0, ...,n — 1. The input/output relation is the pair {'w,t) when t is produced by w. An FSM is 
deterministic if there is a function i5 such that 5{si, wi) = {ti, Si+i) iff (s^, Wi, ti, Si+i) € P. There may be a 
label function L on the states, in some cases. 

Other important models are introduced later. An Extended Finite State Machine (EFSM), introduced 
in section |2.3.3[ assigns variables and their values to states and is a succinct representation of a much larger 
FSM. Transitions assume guards and define updates on the variables. A Biichi automaton, introduced 
in section 2.1.1 generalizes classical automata, i.e. FSM with no output but with accepting states, to 
infinite words. In order to consider probabilistic systems, we introduce Probabilistic Transition Systems and 
Concurrent Probabilistic Systems in section [2. 2| 



2.1 Model checking 

Consider a transition system M = {S, sq, P, L) and a temporal property expressed by a formula ijj of Linear 
Temporal Logic (LTL) or Computation Tree Logic (CTL and CTL*). The Model Checking jtiohlem is to decide 
whether Ai [= tjj, i.e. if the system Ai satisfies the property defined by ijj, and to give a counterexample if 
the answer is negative. 

In linear temporal logic LTL, formulas are composed from the set of atomic propositions using the 
boolean connectives and the main temporal operators X {next time) and U (until). In order to analyze the 
sequential behavior of a transition system A4 , LTL formulas are interpreted over runs or execution paths of 
the transition system M. A path a is an infinite sequence of states {sq, si, . . . , Si, . . .) such that (sj, Si+i) G P 
for all i > 0. We note cr* the path (si, s^+i, . . . ). The interpretation of LTL formulas are defined by: 

• ii p e P then M,a \= p iS p e L{so), 

ip fxij) i^ M,(T \^ (p and M,(j |= -0, 

ifiV if) \^ M,a \^ ip OY M,u \= il^, 
|=XV'iffX,CTi hV', 
1= ip\Jil> iff there exists i > such that A^, cr* \= ijj and for each Q<i<i,M.,a^\=(f, 

The usual auxiliary operators F [eventually) and G [globally) can also be defined: true = p\/ ^p for 
some arbitrary p G P, F-0 = trueXJip and Gip = -^F^ip. 

In Computation Tree Logic CTL*, general formulas combine states and paths formulas. 

1. A state formula is either 

• p if p is an atomic proposition, or 
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• -^F, F AG 01 F y G where F and G are state formulas, or 

• 3ip or V93 where tp is a path formula. 

2. A path formula is either 

• a state formula, or 

• -^ip, if A ^, ip\/ ip, H-ip or iplJi/j where (p and ^ are path formulas. 

State formulas are interpreted on states of the transition system. The meaning of path quantifiers is 
defined by: given A4 and s S S", we say that M,s \— Btp (resp. Ai,s \^ \/ijj) if there exists a path tt starting 
in s which satisfies ip (resp. all paths tt starting in s satisfy ip). 

In CTL, each of the temporal operators X and U must be immediately preceded by a path quantifier. 
LTL can be also considered as the fragment of CTL* formulas of the form V(/9 where (^ is a path formula in 
which the only state subformulas are atomic propositions . It can be shown that the three temporal logics 
CTL*, CTL and LTL have different expressive powers. 

The first model checking algorithms enumerated the reachable states of the system in order to check the 
correctness of a given specification expressed by an LTL or CTL formula. The time complexity of these 
algorithms was linear in the size of the model and of the formula for CTL, and linear in the size of the 
model and exponential in the size of the formula for LTL. The specification can usually be expressed by a 
formula of small size, so the complexity depends in a crucial way on the model's size. Unfortunately, the 
representation of a protocol or of a program with boolean variables by a transition system illustrates the 
state explosion phenomenon: the number of states of the model is exponential in the number of variables. 
During the last twenty years, different techniques have been used to reduce the complexity of temporal logic 
model checking: 

• automata theory and on-thc-fiy model construction, 

• symbolic model checking and representation by ordered binary decision diagram (OBDD), 

• symbolic model checking using propositional satisfiability (SAT) solvers. 

2.1.1 Automata approach 

This approach to verification is based on an intimate connection between linear temporal logic and automata 
theory for infinite words which was first explicitly discussed in |WVS83| . The basic idea is to associate with 
each linear temporal logic formula a finite automaton over infinite words that accepts exactly all the runs 
that satisfy the formula. This enables the reduction of decision problems such as satifiability and model 
checking to known automata-theoretic problems. 

A nondeterministic Biichi automaton is a tuple A — (S, 5, Sq, 6, F), where 

• S is a finite alphabet, 

• 5 is a finite set of states, 

• S'o C S" is a set of initial states, 

• 6 : S X Y, — > 2^ is a transition function, and 

• F C 5* is a set of final states. 

The automaton A is deterministic if \S{s, a)\ — I for all states s G S, for all a G E, and if |S'o| = 1. 

A run of ^ over a infinite word w — agai . . . Oi . . . is a sequence r — sqSi . . . s,; . . . where So S 5*0 and s^+i G 
d{si, Oi) for all i>Q. The limit of a run r — sqSi . . . Si . . . is the set lim{r) = {s\s = Si for infinitely many i\. 
A run r is accepting if lim{r) f] F ^ ^. An infinite word w is accepted by A if there is an accepting run of 
A over w. The language of A, denoted by the regular language L{A), is the set of infinite words accepted 



by A. For any LTL formula if, there exists a nondeterministic Biichi automaton Aip such that the set of 
words satisfymg ip is the regular language L{A)ip and that can be constructed in time and space 0(|(^|. 21*^1). 
Moreover any transition system Ai can be viewed as a Biichi automaton Am ■ Thus model checking can 
be reduced to the comparison of two infinite regular languages and to the emptiness problem for regular 
langu ages jVW 86] : M \= <p iS L{Am) C L{A^) iff L{Am) n L{A^^) = iff L{Am x A^^) = 0. 

In [ VW8 6J , the authors prove that LTL model checking can be decided in time 0(|A^|.2l'''l) and in space 
0{{log\M.\ + |(,c|)^), that is a refinement of the result in |SC85| . which says that LTL model checking is 
PSPACE-complete. One can remark that a time upper bound that is linear in the size of the model and 
exponential in the size of the formula is considered as reasonable, since the specification is usually rather 
short. However, the main problem is the state explosion phenomenon due to the representation of a protocol 
or of a program to check, by a transition system. 

The automata approach can be useful in practice for instance when the transition system is given as a 
product of small components Mi^. . . ^Mk- The model checking can be done without building the product 
automaton, using space 0{{log\M.i\ + • • • + log\M.k\)'^) which is usually much less than the space needed to 
store the product automaton. In |GP VW95] . the authors describe a tableau-based algorithm for obtaining 
an automaton from an LTL formula. Technically, the algorithm translates an LTL formula into a generalized 
Biichi automaton using a depth-first search. A simple transformation of this automaton yields a classical 
Biichi automaton for which the emptiness check can be done using a cycle detection scheme. The result 
is a verification algorithm in which both the transition model and the property automaton are constructed 
on-the-fly during a depth-first search that checks for emptiness. This algorithm is adopted in the model 
checker SPIN EHol03|. 



2.1.2 OBDD approach 

In symbolic model checking jBCM+921 IMcM93] , the transition relation is coded symbolically as a boolean 
expression, rather than expicitly as the edges of a graph. A major breakthrough was achieved by the 
introduction of OBDD's as a data structure for representing boolean expressions in the model checking 
procedure. 

An ordered binary decision diagram (OBDD) is a data structure which can encode an arbitrary relation or 
boolean function on a finite domain. Given a linear order < on the variables, it is a binary decision diagram, 
i.e. a directed acyclic graph with exactly one root, two sinks, labelled by the constants 1 and 0, such that each 
non-sink node is labelled by a variable Xi, and has two outgoing edges which are labelled by 1 (1-edge) and 
(0-edge), respectively. The order, in which the variables appear on a path in the graph, is consistent with the 
variable order <, i.e. for each edge connecting a node labelled by Xi to a node labelled by Xj, we have Xi < Xj. 





Figure 2: Two OBDDs for a function / : {0, 1}^ -^ {0, 1}. 



Let us start with an OBDD representation of the relations _R of A^, the transition relation, and of each 
unary relation P{x) describing states which satisfy the atomic propositions p. Given a CTL formula, one 
constructs by induction on its syntactic structure, an OBDD for the unary relation defining the states where 



it is true, and we can then decide if Ai \= ip. Figure 2.1.2 describes the construction of an OBDD for 



R{x,y) V P{x) from an OBDD for R{x,y) and an OBDD for P(x). Each variable x is decomposed in a 
sequence of boolean variables. In our example a;i,a;2,X3 represent x and similarly for y. The order of the 
variables is xi,X2,X3,yi,y2,y3 in our example. Figure [2. 1 .2| presents a partial decision tree: the dotted line 
corresponds to a;^ = and the standard line corresponds to Xi = 1. The tree is partial to make it readable, 
and missing edges lead to 0. The main drawback is that the OBDD can be exponentially large, even for 
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Figure 3: The construction of an OBDD for R{x, y) V P{x). 



simple formulas [Bry91'. The good choice of the order on the variables is important, as the size of the OBDD 
may vary exponentially if we change the order. 



2.1.3 SAT approach 

Symbolic model checking and symbolic reachability analysis can be reduced to the satisfiability problem for 
propositional formulas BCCZ99a, ABEOOa . These reductions will be explained in the section 4.1: bounded 
and unbounded model checking. In the following, we recall the quest for efficient satisfiability solvers which 
has been the subject of an intensive research during the last twenty years. 

Given a propositional formula which is presented in a Conjunctive Normal Form (CNF), the goal is 
to find a positive assignment of the formula. Recall that, a CNF is a conjunction of one or more clauses 
Ci A C2 A C3 A . . ., where each clause is a disjunction of one or more literals, Ci = xi V X2 V is V xy, 
C2 — x^iW x-! , Cj, = . . .. A literal is either the positive or the negative occurrence of a propositional variable, 
for instance X2 and X2 are the two literals for the variable X2- 

Due to the NP-completeness of SAT, it is unlikely that there exists any polynomial time solution. How- 
ever, NP-completeness does not exclude the possibility of finding algorithms that are efficient enough for 
solving many interesting SAT instances. This was the motivation for the development of several successful 
algorithms |ZM02j . 

An original important algorithm for solving SAT, due to [DP60J . is based on two simplification rules 
and one resolution rule. As this algorithm suffers from a memory explosion, |DLL62j proposed a modified 
version (DPLL) which performs a branching search with backtracking, in order to reduce the memory space 
required by the solver. 

|MSS96| proposed an iterative version of DPLL, that is a branch and search algorithm. Most of the 
modern SAT solvers are designed in this manner and the main components of these algorithms are: 



• a decision process to extend the current assignment to an unassigned variable; this decision is usually 
based on branching heuristics, 

• a deduction process to propagate the logical consequences of an assignment to all clauses of the SAT 
formula; this step is called Boolean Constraint Propagation (BCP), 

• a conflict analysis which may lead to the identification of one or more unsatisfied clauses, called 
conflicting clauses, 

• a backtracking process to undo the current assignment and to try another one. 

In a SAT solver, the BCP step is to propagate the consequences of the current variable assignment to 
the clauses. In CHAFF [MMZ+Ol , Moskewicz et al. proposed a BCP algorithm called two-literal watching 



with lazy update. Since the breakthrough of CHAFF, most effort in the design of efficient SAT solvers has 
been focused on efficient BCP, the heart of all modern SAT solvers. 

An additional technique named Random restart was proposed to cope with the following phenomenon: 
two instances with the same clauses but different variable orders may require different times by a SAT solver. 
Experiments show that a random restart can increase the robustness of SAT solvers and this technique is 
applied in modern SAT solvers such as RSTART |PD07j . TiniSAT |Hua07| and PicoSAT |Bie08| . This 
technique, for example the nested restart scheme used by PicoSAT, is inspired by the work of M. Luby et 
al. |LSZ93j . 

Another significant extension of DPLL is clause learning: when there is a conflict after some propagation, 
and there are still some branches to be searched, the cause of the conflict is analysed and added as a new 
clause before backtracking and continuing the search |BKS03j . Various learning schemes have been proposed 
[AS09| to derive the new clauses. Combined with non chronological backtracking and random restart these 
techniques are currently the basis of modern SAT-solvers, and the origin of the spectacular increase of their 
performance. 

2.2 Verification of probabilistic systems 

In this section, we consider systems modeled either as finite discrete time Markov chains or as Markov 
models enriched with a nondeterministic behavior. In the following, the former systems will be denoted 
by probabilistic sytems and the latter by concurrent probabilistic sytems. A Discrete Time Markov Chain 
(DTMC) is a pair {S, M) where S* is a finite or countable set of states and M : S x S —>■ [0, 1] is the stochastic 
matrix giving the transition probabilities, i.e. for all s G S, J^tes -^(^'^) ~ ^- ^^ ^^^ following, the set of 
states S is finite. 

Definition 1 A probabilistic transition system (PTS) is a structure A4p — {S, sq, Af, L) given by a Discrete 
Time Markov chain (S, M) with an initial state sq and a function L : 5 — > V{P) labeling each state with a 
set of atomic propositions in P. 

A path cr is a finite or infinite sequence of states (sq, si, . . . , Si, ■ . . ) such that P{si, s^+i) > for all 
j > 0. We denote by Path{s) the set of paths whose first state is s. For each structure A4 and state s, it is 
possible to define a probability measure Prob on the set Path{s). For any finite path w = (sq, si, . . . , s„), 
the measure is defined by: 

n 

Prob{{a : cr is a path with prefix tt}) = 1 I M{si-i, Si) 

This measure can be extended uniquely to the Borel family of sets generated by the sets 
{cr : TT is a prefix of a"\ where tt is a finite path. In [VarSS^ , it is shown that for any LTL formula V'j 
probabilistic transition system M. and state s, the set of paths \a : ctq = s and M.^a h" "0} is measurable. 
We denote by Prob\i\j\ the measure of this set and by Probk\i\)\ the probability measure associated to the 
probabilistic space of execution paths of finite length k. 



2.2.1 Qualitative verification 

We say that a probabilistic transition sytem A4p satisfies the formula ■0 if Prob[ilj] = 1, i.e. if almost all 
paths in M, whose origin is the initial state, satisfy ip. The first application of verification methods to 
probabilistic systems consisted in checking if temporal properties are satisfied with probability 1 by a finite 
discrete time Markov chain or by a concurrent probabilistic sytem. |Var85) presented the first method to 
verify if a linear time temporal property is satisfied by almost all computations of a concurrent probabilistic 
system. However, this automata-theoretic method is doubly exponential in the size of the formula. 

The complexity was later addressed in |CY95j . A new model checking method for probabilistic systems 
was introduced, whose complexity was polynomial in the size of the system and exponential in the size of 
the formula. For concurrent probabilistic systems they presented an automata-theoretic approach which 
improved on Vardi's method by a single exponential in the size of the formula. 

2.2.2 Quantitative verification 

The |CY95j method allows to compute the probability that a probabilistic system satisfies some given linear 
time temporal formula. 

Theorem 1 (\CY93^ ) The satisfaction of a LTL formula (j) hy a probabilistic transition sytem A4p can be 
decided in time linear in the size of Aip and exponential in the size of (j), and in space polylogarithmic in the 
size of Mp and polynomial in the size of (p. The probability Prob[(l)] can be computed in time polynomial in 
size of Mp and exponential in size of (p. 

A temporal logic for the specification of quantitative properties, which refer to a bound of the probability 
of satisfaction of a formula, was given in HJ94 . The authors introduced the logic PCTL, which is an exten- 
sion of branching time temporal logic CTL with some probabilistic quantifiers. A model checking algorithm 
was also presented: the computation of probabilities for formulas involving probabilistic quantification is 
performed by solving a linear system of equations, the size of which is the model size. 

A model checking method for concurrent probabilistic systems against PCTL and PCTL* (the standard 
extension of PCTL) properties is given in [BdA95, . Probabilities are computed by solving an optimisation 
problem over system of linear inequalities, rather than linear equations as in jH J94j . The algorithm for the 
verification of PCTL* is obtained by a reduction to the PCTL model checking problem using a transforma- 
tion of both the formula and the probabilistic concurrent system. Model checking of PCTL formulas is shown 
to be polynomial in the size of the system and linear in the size of the formula, while PCTL* verification is 
polynomial in the size of the system and doubly exponential in the size of the formula. 

In order to illustrate space complexity problems, we mention the main model checking tool for the 
verification of quantitative properties. The probabilistic model checker PRISM dAKN+OOl IIIKNP06] was 



designed by the Kwiatkowska's team and allows to check PCTL formulas on probabilistic or concurrent 
probabilistic systems. This tool uses extensions of OBDDs called Multi-Terminal Binary Decision Diagrams 
(MTBDDs) to represent Markov transition matrices, and classical techniques for the resolution of linear 
systems. Numerous classical protocols represented as probabilistic or concurrent probabilistic systems have 
been successfully verified by PRISM. But experimental results are often limited by the exponential blow up 
of space needed to represent the transition matrices and to solve linear systems of equations or inequations. 
In this context, it is natural to ask the question: can probabilistic verification be efficiently approximated? 
We study in Section [4.5| some possible answers for probabilistic transition systems and linear time temporal 
logic. 

2.3 Model-based testing 

Given some executable implementation under test and some description of its expected behavior, the lUT is 
submitted to experiments based on the description. The goal is to (partially) check that the lUT is conforming 
to the description. As we explore links and similarities with model checking, we focus on descriptions defined 
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in terms of finite and infinite state machines, transitions systems, and automata. The corresponding testing 
methods are called Model-based Testing. 

Model-based testing has received a lot of attention and is now a well established discipline (see for instance 
[LY96[ IBTOli iBJK+OSj ). Most approaches have focused on the deterministic derivation from a finite model 
of some so-called checking sequence, or of some complete set of test sequences, that ensure conformance of 
the lUT with respect to the model. However, in very large models, such approaches are not practicable and 
some selection strategy must be applied to obtain test sets of reasonable size. A popular selection criterion 
is the transition coverage. Other selection methods rely on the statement of some test purpose or on random 
choices among input sequences or traces. 

2.3.1 Testing based on finite state machines 

As in |LY96| . we first consider testing methods based on deterministic FSMs: instead of T = (5*, sq, I, O, R) 
where RCSxIxOxS,we have J- = {S, I, O, S, A), where 5 and A are functions from S x I into S, and 
from S X I into O, respectively. There is not always an initial state. Functions 6 and A can be extended in 
a canonic way to sequences of inputs: 6* is from S x I* into S'*and A* is from S x I* into O* . 

The testing problem addressed in this subsection is: given a deterministic specification FSM A, and an 
lUT that is supposed to behave as some unknown deterministic FSM B, how to test that B is equivalent 
to A via inputs submitted to the lUT and outputs observed from the lUT? The specification FSM must 
be strongly connected, i.e., there is a path between every pair of states: this is necessary for designing test 
experiments that reach every specified state. 

Equivalence of FSMs is defined as follows. Two states Si and Sj are equivalent if and only if for every 
input sequence, the FSMs will produce the same output sequence, i.e., for every input sequence a^ A* (si, a) = 
X*{sj, a). J- and J-' are equivalent if and only for every state in J- there is a corresponding equivalent state in 
J-' ^ and vice versa. When T and J-' have the same number of states, this notion is the same as isomorphism. 
Given an FSM, there are well-known polynomial algorithms for constructing a minimized (reduced) FSM 
equivalent to the given FSM, where there are no equivalent states. The reduced FSM is unique up to 
isomorphism. The specification FSM is supposed to be reduced before any testing method is used. 

Any test method is based on some assumption on the lUT called testability hypotheses. An example of 
a non testable lUT would be a "demonic" one that would behave well during some test experiments and 
change its behavior afterwards. Examples of classical testability hypotheses, when the test is based on finite 
state machine descriptions, are: 

• The lUT behaves as some (unknown) finite state machine. 
The implementation machine docs not change during the experiments. 

• It has the same input alphabet as the specification FSM. 
It has a known number of states greater or equal to the specification FSM. 



• 



• 



This last and strong hypothesis is necessary to develop testing methods that reach a conclusion after a 
finite number of experiments. In the sequel, as most authors, we develop the case where the lUT has the 
same number of states as the specification FSM. Then we give some hints on the case where it is bigger. 

A test experiment based on a FSM is modelled by the notion of checking sequence, i. e. a finite sequence 
of inputs that distinguishes by some output the specification FSM from any other FSM with at most the 
same number of states. 

Definition 2 Let A be a specification FSM with n states and initial state sq . A checking sequence for A is 
an input sequence Ocheck such that for every FSM B with initial state Sq, the same input alphabet, and at 
most n states, that is not isomorphic to A, X*g{sQ,acheck) 7^ ^*a{-^Ot '^ check)- 

The complexity of the construction of checking sequences depends on two important characteristics of 
the specification FSM: the existence of a reliable reset that makes it possible to start the test experiment 
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from a known state, and the existence of a distinguishing sequence a, which can identify the resulting state 
after an input sequence, i.e. such that for every pair of distinct states Si, Sj, X*{si,a) ^ X*{sj,a). 

A reliable reset is a specific input symbol that leads an FSM from any state to the same state: for 
every state s, 6{s, reset) = s^- For FSM without reliable reset, the so-called homing sequences are used 
to start the checking sequence. A homing sequence is an input sequence ah such that, from any state, the 
output sequence produced by an determines uniquely the arrival state. For every pair of distinct states 
Si, Sj, A*(si, CT/i) = \*{sj,ah) implies 5*{si,Uh) = S*{sj,ah)- Every reduced FSM has an homing sequence of 
polynomial length, constructible in polynomial time. 

The decision whether the behavior of the lUT is satisfactory, requires to observe the states of the lUT 
either before or after some action. As the lUT is a running black box system, the only means of observation is 
by submitting other inputs and collecting the resulting outputs. Such observations are generally destructive 
as they may change the observed state. 

The existence of a distinguishing sequence makes the construction of a checking sequence easier: an 
example of a checking sequence for a FSM A is a sequence of inputs resulting in a trace that traverses once 
every transition followed by this distinguishing sequence to detect for every transition both output errors 
and errors of arrival state. 

Unfortunately deciding whether a given FSM has a distinguishing sequence is PSPACE-complete with 
respect to the size of the FSM (i.e. the number of states). However, it is polynomial for adaptative 
distinguishing sequences (i.e input trees where choices of the next input are guided by the outputs of the 
lUT), and it is possible to construct one of quadratic length. For several variants of these notions, see |LY96j . 

Let p the size of the input alphabet. For an FSM with a reliable reset, there is a polynomial time algorithm, 
in 0{p.n^), for constructing a checking sequence of polynomial length, also in 0{p.n'^) |Vas731 [Cho78| . For 
an FSM with a distinguishing sequence there is a deterministic polynomial time algorithm to construct a 
checking sequence |Hen641 IKHF90J of length polynomial in the length of the distinguishing sequence. 

In other cases, checking sequences of polynomial length also exist, but finding them requires more involved 
techniques such as randomized algorithms. More precisely, a randomized algorithm can construct with high 
probability in polynomial time a checking sequence of length 0{p.n^ + p'.n^. logn), with p' = min{p,n). 
The only known deterministic complexity of producing such sequences is exponential either in time or in the 
length of the checking sequence. 

The above definitions and results generalize to the case where FSM B has more states than FSM A. 
The complexity of generating checking sequences, and their lengths, are exponential in the number of extra 
states. 

2.3.2 Non determinism 

The concepts presented so far are suitable when both the specification FSM and the lUT are deterministic. 
Depending on the context and of the authors, a non deterministic specification FSM A can have different 
meanings: it may be understood as describing a class of acceptable deterministic implementations or it can 
be understood as describing some non deterministic acceptable implementations. In both cases, the notion 
of equivalence of the specification FSM A and of the implementation FSM B is no more an adequate basis 
for testing. Depending of the authors, the required relation between a specification and an implementation is 
called the "satisfaction relation" {B satisfies A) or the "conformance relation" {B conforms to A). Generally 
it is not an equivalence, but a preorder (see |Tre92[ IGJ98[ IBTOl) among many others) . 

A natural definition for this relation could be the so-called "trace inclusion" relation: any trace of the 
implementation must be a trace of the specification. Unfortunately, this definition accepts, as a conforming 
implementation of any specification, the idle implementation, with an empty set of traces. Several more 
elaborated relations have been proposed. The most known are the conf relation, between Labelled Transition 
Systems |Bri88j and the ioco relation for Input-Output Transition Systems |Tre96j . The intuition behind 
these relations is that when a trace a (including the empty one) of a specification A is executable by some 
lUT B, after a, B can be idle only if A may be idle after a, else B must perform some action performable 
by A after a. For Finite State Machines, it can be rephrased as: an implementation FSM B conforms to a 
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specification FSM A if all its possible responses to any input sequence could have been produced by A, a 
response being the production of an output or idleness. 

Not surprisingly, non dctcrniinisni introduces major complications when testing. Checking sequences are 
no more adequate since some traces of the specification FSM may not be executable by the lUT. One has to 
define adaptative checking sequences (which, actually, are covering trees of the specification FSM) in order 
to let the lUT choose non-deterministically among the allowed behaviors. 

2.3.3 Symbolic traces and constraints solvers 

Finite state machines (or finite transition systems) have a limited description power. In order to address 
the description of realistic systems, various notions of Extended Finite State Machines (EFSM) or symbolic 
labelled transition systems (SLTS) are used. They are the underlying semantic models in a number of 
industrially significant specification techniques, such as LOTOS, SDL, Statecharts, to name just a few. To 
make a long story short, such models are enriched by a set of typed variables that are associated with the 
states. Transitions are labelled as in FSM or LTS, but in addition, they have associated guards and actions, 
that are conditions and assignments on the variables. In presence of such models, the notion of a checking 
sequence is no more realistic. Most EFSM-based testing methods derive some test set from the EFSM, that 
is a set of input sequences that ensure some coverage of the EFSM, assuming some uniform behavior of the 
lUT with respect to the conditions that occur in the EFSM. 

More precisely, an Extended Finite State Machine (EFSM) is a structure (5, sq, /, IP, O, T, V, Hq) where 
5" is a finite set of states with initial state sq, / is a set of input values and IP is a set of input parameters 
(variables), O is a set of output values, T is a finite set of symbolic transitions, y is a finite list of variables 
and vq is a list of initial values of the variables. Each association of a state and variable values is called 
a configuration. Each symbolic transition t in T is a 6-tuple: t — (st, s^,it,Ot,Gt, At) where St,St are 
respectively the current state, and the next state of t; it is an input value or an input parameter; ot is an 
output expression that can be parametrized by the variables and the input parameter. Gt is a predicate 
(guard) on the current variable values and the input parameter and At is an update action on the variables 
that may use values of the variables and of the input. Initially, the machine is in an initial state sq with 
initial variable values: wo. 

An action v :^ v + n indicates the update of the variable v. Figure l4] gives a very simple example of such 
an EFSM. It is a bounded counter which receives increment or decrement values. There is one state variable 
V whose domain is the integer interval [0..10]. The variable v is initialized to 0. The input domain / is Z. 
There is one integer input parameter n. When an input would provoke an overflow or an underflow of v, it 
is ignored and v is unchanged. Transitions labels follows the following syntax: 

? < input value or parameter > /! < output expression > / < guard > / < action > 

An EFSM operates as follows: in some configuration, it receives some input and computes the guards 
that are satisfied for the current configuration. The satisfied guards identify enabled transitions. A single 
transition among those enabled is fired. When executing the chosen transition, the EFSM 

• reads the input value or parameter value it, 

• updates the variables according to the action of the transition, 

• moves from the initial to the final state of the transition, 

• produces some output , which is computed from the values of the variables and of the input via the 
output expression of the transition. 

Transitions are atomic and cannot be interrupted. Given an EFSM, if each variable and input parameter 
has a finite number of values (variables for booleans or for intervals of finite integers, for example), then there 
is a finite number of configurations, and hence there is a large equivalent (ordinary) FSM with configurations 
as states. Therefore, an EFSM with finite variable domains is a succinct representation of an FSM. Generally, 
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Figure 4: Example of an EFSM: counter with increment and decrement values. 

constructing this FSM is not easy because of the reachability problem, i.e. the issue of determining if a 
configuration is reachable from the initial state. It is undecidable if the variable domains are infinite and 
PSPACE-complete otherwise^ 

A symbolic trace ii, . . . , i„ of an EFSM is a sequence of symbolic transitions such that Stj^ = sq and for 
i = 1, . . . 71 — 1, Sj. = St;^j^. A trace predicate is the condition on inputs which ensures the execution of a 
symbolic trace. Such a predicate is built by traversing the trace <i, . . . , <„ in the following way: 

• the initial index of each variable x is 0, and for each variable x there is an equation xq = Wq, 

• for i = 1 . . . n, given transition ti with guard G^, and action Ai: 

— guard Gi is transformed into the formula Gi where each variable of G has been indexed by its 
current index, and the input parameter (if any) is indexed by i, 

— each assignment in Ai of an expression expr to some variable x is transformed into an equation 
Xk+i — expr^ where k is the current index of x and expr^ is the expression expr where each 
variable is indexed by its current index, and the input parameter (if any) is indexed by i, 

— the current indexes of all assigned variables are incremented, 

• the trace predicate is the conjunction of all these formulae. 

A symbolic trace is feasible if its predicate is satisfiable, i.e. there exist some sequence of input values 
that ensure that at each step of the trace, the guard of the symbolic transition is true. Such a sequence of 
inputs characterizes a trace of the EFSM. A configuration is reachable if there exists a trace leading to it. 

EFSM testing methods must perform reachability analysis: to compute some input sequence that exercises 
a feature (trace, transition, state) of a given EFSM, a feasible symbolic trace leading to and covering 
this feature must be identified and its predicate must be solved. Depending on the kind of formula and 
expression allowed in guards and actions, different constraint solvers may be used [CGK"*" 1 ll ITGMJT] . Some 
tools combine them with SAT-solvers, model checking techniques, symbolic evaluation methods including 
abstract interpretation, to eliminate some classes of clearly infeasible symbolic traces. 

The notion of EFSM is very generic. The corresponding test generation problem is very similar to test 
generation for programs in general. The current methods address specific kinds of EFSM or SLTS. There 
are still a lot of open problems to improve the levels of generality and automation. 



^As said above, there are numerous variants of the notions of EFSM and SLTS. The complexity of their analysis (and thus 
of their use as a basis for black box testing) is strongly dependent on the types of the variables and of the logic used for the 
guards. 
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2.3.4 Classical methods in probabilistic and statistical testing 

Drawing test cases at random is an old idea, which looks attractive at first sight. It turns out that it is 
difficult to estimate its detection power. Strong hypotheses on the JUT, on the types and distribution of 
faults, are necessary to draw conclusions from such test campaigns. Depending on authors and contexts, 
testing methods based on random selection of test cases are called: random testing, or probabilistic testing or 
statistical testing. These methods can be classified into three categories : those based on the input domain, 
those based on the environment, and those based on some knowledge of the behavior of the lUT. 

In the first case, classical random testing (as studied in |DN81|. IDN84J ) consists in selecting test data 
uniformly at random from the input domain of the program. In some variants, some knowledge on the input 
domain is exploited, for instance to focus on the boundary or limit conditions of the software being tested 
|Rei97llNtaOT] . 

In the second case, the selection is based on an operational profile, i.e. an estimate of the relative 
frequency of inputs. Such testing methods are called statistical testing. They can serve as a statistical 
sampling method to collect failure data for reliability estimation (for a survey see [MFI+96I V 

In the third case, some description of the behavior of the lUT is used. In |TFW9l] , the choice of the 
distribution on the input domain is guided either by some coverage criteria of the program and they call 
their method structural statistical testing, or by some specification and they call their method functional 
statistical testing. 

Another approach is to perform random walks jAldQl] in the set of execution paths or traces of the 
lUT. Such testing methods were developed early in the area of communication protocols |Wes891 rMP94] . In 
[Wes89] , West reports experiments where random walk methods had good and stable error detection power. 
In [MP94 , some class of models is identified, namely those where the underlying graph is symmetric, which 
can be efficiently tested by random walk exploration: under this strong condition, the random walk converges 
to the uniform distribution over the state space in polynomial time with respect to the size of the model. A 
general problem with all these methods is the impossibility, except for some very special cases, to assess the 
results of a test campaign, either in term of coverage or in term of fault detection. 

3 Methods for approximation 

In this section we classify the different approximations introduced in model checking and testing in two 
categories. Methods which approximate decision problems, based on some parameters, and methods which 
study approximate versions of the decision problems. 

1. Approximate methods for decision, counting and learning problems. The goal is to define useful 
heuristics on practical inputs. SAT is the typical example where no polynomial algorithm exists 
assuming P ^ NP, but where useful heuristics are known. The search for abstraction methods by 
successive refinements follows the same approach. 

2. Approximate versions of decision and learning problems relax the decision by introducing some error 
parameter e. In this case, we may obtain efficient randomized algorithms, often based on statistics for 
these new approximate decision problems. 

Each category is detailed in subsections below. First, we introduce the classes of efficient algorithms we 
will use to elaborate approximation methods. 

3.1 Randomized algorithms and complexity classes 

The efficient algorithms we study are mostly randomized algorithms which operate in polynomial time. They 
use an extra instruction, flip a coin, and we obtain or 1 with probability ^. As we make n random fiips, 
the probabilistic space il consists of all binary sequences of length n, each with probability :^. We want to 
decide if a; e i C S*, such that the probability of getting the wrong answer is less than ^ for some fixed 
constant c, i.e. exponentially small. 
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Definition 3 An algorithm A is Bounded-error Probabilistic Polynomial-tinic (BPP), for a language L C 
E* if A is in polynomial tim,e and: 

• if X € L then A accepts x with probability greater then 2/3, 

• if X ^ L then A rejects x with probability greater then 2/3. 

The class BPP consists of all languages L which admit a bounded-error probabilistic polynomial time algo- 
rithm. 

In this definition, we can replace 2/3 by any value strictly greater than 1/2, and obtain an equivalent 
definition. In some cases, 2/3 is replaced by 1/2 + e or by 1 — (5 or by 1 — 1/n''. If we modify the second 
condition of the previous defintion by: ii x ^ L then A rejects x with probability 1, we obtain the class RP, 
Randomized Polynomial time. 

We recall the notion of a p-predicate, used to define the class NP of decision problems which are verifiable 
in polynomial time. 

Definition 4 A p-predicate R is a binary relation between words such that there exist two polynomials 
p, q such that: 

• for a^/ X, y G S*, i?(x, y) implies that \ y |< p(| x |); 

• for all X, y G S* , i?(x,y) is decidable in time q{\ x |). 

A decision problem A is in the class NP if there is a p-predicate R such that for all x, x € A\E 3yR{x, y). 
Typical examples are SAT for clauses or CLIQUE for graphs. For SAT, the input x is a set of clauses, y is 
a valuation and R{x,y) if y satisfies x. For CLIQUEj., the input a; is a graph, j/ is a subset of size k of the 
nodes and i?(x, y) if j/ is a clique of x, i.e. if all pairs of nodes in y are connected by an edge. 

One needs a precise notion of approximation for a counting function F : E* — > TV using an efficient 
randomized algorithm whose relative error is bounded by e with high probability, for all e. It is used in 
section 



4.5.3 to approximate probabilities. 



Definition 5 An algorithm A is a Polynomial-time Randomized Approximation Scheme (PRAS) for a 
function i^ : E* — > N if for every e and x, 

PrM(x,e)e[(l-e).F(x),(l -(-£). F(x)]} > ^ 

and ^(x, e) stops in polynomial time m | x | . The algorithm A is a Fully Polynomial time Randomized 
Approximation Schema (FPRAS), if the time of computation is also polynomial in l/e. The class PRAS 
(resp. FPRAS) consists of all functions F which admits a PRAS (resp. FPRAS) . 

If the algorithm A is deterministic, one speaks of an PAS and of a FPAS. A PRAS{S) (resp. 
FPRAS{5)), is an algorithm A which outputs a value A{x.,e,6) such that: 

Pr{^(x,£,<5)e[(l-£).F(x),(l + £).F(x)]} > 1-^ 

and whose time complexity is also polynomial in log (1/(5). The error probability is less than S in this model. 
In general, the probability of success can be amplified from 2/3 to 1 — (5 at the cost of extra computation of 
length polynomial in \og{l/S). 

Definition 6 A counting function F is in the class ^P if there exists a p-predicate R such that for all x, 
Fix)^\{y:{x,y)eR}\. 

If A is an NP problem, i.e. the decision problem on input x which decides if there exists y such that R{x, y) 
for a p-predicate R, then #v4 is the associated counting function, i.e. #A{x) =| {y : {x,y) & R} \. The 
counting problem ffSAT is ^^P-complete and not approximable (modulo some complexity conjecture). On 
the other hand fj^DNF is also #P-complete but admits an FPRAS |KL83) . 
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3.2 Approximate methods for satisfiability, equivalence, counting and learning 

Satisfiability decides given a model Ai and a formula ip, whether A4 satisfies a formula ■0. Equivalence 
decides given two models M and A4' , whether they satisfy the same class of formulas. Counting associates 
to a formula ip, the number of models A4 which satisfy a formula ip. Learning takes a black box which 
defines an unknown function / and tries to find from samples Xi,yi — f{xi). 

3.2.1 Approximate satisfiability and abstraction 

To verify that a model M satisfies a formula ^ , abstraction can be used for constructing approximations of M 
that are sufficient for checking ip. This approach goes back to the notion of Abstract Interpretation, a theory 
of semantic approximation of programs introduced by Cousot et al.fCCTT'], which constructs elementary 
embeddingfP] that suffice to decide properties of programs. A classical example is multiplication, where 
modular arithmetic is the basis of the abstraction, ft has been applied in static analysis to find sound, finite, 
and approximate representations of a program. 

In the framework of model checking, reduction by abstraction consists in approximating infinite or very 
large finite transition systems by finite ones, on which existing algorithms designed for finite verification are 
directly applicable. This idea was first introduced by Clarke et al. [EMCL94] . Graf and Saidi [GS97| have 
then proposed the predicate abstraction method where abstractions are automatically obtained, using decision 
procedures, from a set of predicates given by the user. When the resulting abstraction is not adequate for 
checking -0, the set of predicates must be revised. This approach by abstraction refinement has been recently 
systematized, leading to a quasi automatic abstraction discovery method known as Counterexample- Guided 
Abstraction Refinement (CEGAR) |CGJ"'"03 . It relies on the iteration of three kinds of steps: abstraction 



construction, model checking of the abstract model, abstraction refinement, which, when it terminates, states 
whether the original model satifies the formula. 

This section starts with the notion of abstraction used in model checking, based on the pioneering paper 
by Clarke et al.. Then, we present the principles of predicate abstraction and abstraction refinement. 

In |EMCL94 ]. Clarke and al. consider transition systems A4 where atomic propositions are formulas 
of the form v — d, where w is a variable and d is a constant. Given a set of typed variable declarations 
fi : Ti, . . . , w„ : r„, states can be identified with n-tuples of values for variables, and the labeling function 
L is just defined by L{s) = {s}. On such systems, abstractions can be defined by a surjection for each 
variable into a smaller domain. It reduces the size of the set of states. Transitions arc then stated between 
the resulting equivalence classes of states as defined below. 

Definition 7 (\EMCL94^ ) Let M be a transition system, with set of states S , transition relation R, and 
a set of initial states I C S. An abstraction for A4 is a surjection h : S ^ S. A transition system 
M = {S,I,R,L) approximates M with respect to h (M Qh M for short) if h{I) C / and {h{s),h{s')) e R 
for all (s, s') e R. 

Such an approximation is called an over approximation and is explicitly given in |EMGL9^ from a given 
logical representation of A4 . 

Now, let Ai be an approximation of Ai. Suppose that A4 \= Q. What can we conclude on the concrete 
model Ail First consider the following transformations C and V between CTL* formulas on M and their 
approximation on At. These transformations preserve boolean connectives, path quantifiers, and temporal 
operators, and act on atomic propositions as follows: 

C{v = d) = y iv = d), V{v = d) = {v = h{d)). 

d:h(d)=d 

Denote by VCTL* and 3CTL* the universal fragment and the existential fragment of CTL* . The following 
theorem gives correspondences between models and their approximations. 



^Lct U and V be two structures with domain A and B. In logie, an elementary embedding of U into V is a function f : A ^ B 
such that for all formulas ip{xi, ..., x„) of a logic, for all elements ai, ..., a„ a A, U \= <^[ai, ..., an] iff V |= ip[f{ai), ..., /(nu)]. 
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Theorem 2 ( |EMCL94p Let Ai = {S,I,R,L) be a transition system. Let h : S ^ S be an abstraction 
for M, and let M be such that Ai C/i M. Let Q be a VCTL* formula on M , and Q' be a 3CTL* formula 
on M . Then 

7W h =^ -A^ h C(e) and M\=Q' =^ M'^V{Q'). 

Abstraction can also be used when the target structure does not follow the original source signature. 
In this case, some specific new predicates define the target structure and the technique has been called 
predicate abstraction by Graf et al. |GS97j . The analysis of the small abstract structure may suffice to prove 
a property of the concrete model and the authors define a method to construct abstract state graphs from 
models of concurrent processes with variables on finite domains. In these models, transitions are labelled 
by guards and assignments. The method starts from a given set of predicates on the variables. The choice 
of these predicates is manual, inspired by the guards and assignments occurring on the transitions. The 
chosen predicates induce equivalence classes on the states. The computation of the successors of an abstract 
state requires theorem proving. Due to the number of proofs to be performed, only relatively small abstract 
graphs can be constructed. As a consequence, the corresponding approximations are often rather coarse. 
They must be tuned, taking into account the properties to be checked. 
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Figure 5: CEGAR:Counterexample-Guided Abstraction Refinement. 

We now explain how to use abstraction refinement in order to achieve WCTL* model checking: for a 
concrete structure Ai and an VCTL* formula ■0, we would like to check if M ^ ^. The methodology of the 
counterexample-guided abstraction refinement jCGJ"'"03j consists in the following steps: 

• Generate an initial abstraction A4. 

• Model check the abstract structure. If the check is affirmative, one can conclude that A4 \^ ip; 
otherwise, there is a counterexample to jM |= "tjj. To verify if it is a real counterexample, one can check 
it on the original structure; if the answer is positive, it is reported it to the user; if not, one proceeds 
to the refinement step. 

• Refine the abstraction by partitioning equivalence classes of states so that after the refinement, the new 
abstract structure does not admit the previous counterexample. After refining the abstract structure, 
one returns to the model checking step. 

The above approaches are said to use over approximation because the reduction induced on the models 
introduces new paths, while preserving the original ones. A notion of under approximation is used in bounded 
model checking where paths are restricted to some finite lengths. It is presented in section 4.1 Another 



approach using under approximation is taken in [MS0 7] for the class of models with input variables. The 
original model is coupled with a well chosen logical circuit with m < n input variables and n outputs. The 
model checking of the new model may be easier than the original model checking, as fewer input variables 
are considered. 
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3.2.2 Uniform generation and counting 

In this section we describe the hnk between generating elements of a set S and counting the size of S, 
first in the exact case and then in the approximate case. The exact case is used in section |4.4.2| and the 
approximate case is later used in section [4. 5. 3| to approximate probabilities. 

Exact case. Let S'„ be a set of combinatorial objects of size n. There is a close connection between 
having an explicit formula for | Sn \ and a uniform generator for objects in Sn- Two major approaches 
have been developed for counting and drawing uniformly at random combinatorial structures: the Markov 
Chain Monte-Carlo approach (see e.g. the survey |JS96j ) and the so-called recursive method, as described in 
[FZC94] and implemented in [Thi04J. Although the former is more general in its applications, the latter is 
particularly efficient for dealing with the so-called decomposable combinatorial classes of Structures, namely 
classes where structures are formed from a set Z of given atoms combined by the following constructions: 

+, X , Seq, PSet, MSet, Cyc 

respectively corresponding to disjoint union, Cartesian product, finite sequence, multiset, set, directed cycles. 
It is possible to state cardinality constraints via subscripts (for instance Seq<3). These structures are called 
decomposable structures. The size of an object is the number of atoms it contains. 

Example 1 Trees : 

• The class B of binary trees can be specified by the equation B ^ Z + {B x B) where Z denotes a fixed 
set of atoms. 

• An example of a .structure in B is (Z x {Z x Z)). Its size is 3. 

• For non empty ternary trees one could write T — Z + Seq^3(T) 

The enumeration of decomposable structures is based on generating functions. Let C„ the number of 
objects of C of size n, and the following generating function: 



C(z) - Yl ^"^^ 



n<0 

Decomposable structures can be translated into generating functions using classical results of combinato- 
rial analysis. A comprehensive dictionary is given in |FZC94j . The main result on counting and random 
generation of decomposable structures is: 

Theorem 3 Let C be a decomposable combinatorial class of structures. Then the counts {Cj \j — . . .n} 
can be computed in 0(n^'^^) arithmetic operations, where e is a constant less than 1. In addition, it is 
possible to draw an element of size n uniformly at random in 0{nlogn) arithmetic operations in the worst 
case. 

A first version of this theorem, with a computation of the counting sequence {Cj|j — . . .n} in O(n^) was 
given in [FZC94J . The improvement to 0{n^~^^) is due to van der Hoeven |vdH02j . 

This theory has led to powerful practical tools for random generation [Thi04| . There is a preprocessing 
step for the construction of the {Cj\j = 0...n} tables . Then the drawing is performed following the 
decomposition pattern of C, taking into account the cardinalities of the involved sub-structures. For instance, 
in the case of binary trees, one can uniformly generate binary trees of size n -f 1 by generating a random 
k < n, with probability 
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where Bk is the set of binary trees of size k. A tree of size n + 1 is decomposed into a subtree on the 
left side of the root of size k and into a subtree on the right side of the root of size n — k. One recur- 
sively applies this procedure and generates a binary tree with n atoms following a uniform distribution on Bn- 

Approximate case. In the case of a hard counting problem, i.e. when | Sn \ does not have an explicit 
formula, one can introduce a useful approximate version of counting and uniform generation. Suppose the 
objects are witnesses of a p-predicate, i.e. they can be recognized in polynomial time. 

Approximate counting S can be reduced to approximate uniform generation oi y € S and conversely ap- 
proximate uniform generation can be reduced to approximate counting, for self-reducible sets. Self-reducible 
sets guarantees that a solution for an instance of size n depends directly from solutions for instances of size 
n — 1. For example, in the case of SAT, a valuation on n variables pi, ...,Pn on an instance x is either a 
valuation of an instance xi of size n — 1 where p„ = 1 or a valuation of an instance xo of size n — 1 where 
Pn = 0. Thus the p-predicate for SAT is a self-reducible relation. 

To reduce approximate counting to approximate uniform generation, let So- be the set S where the first 
I 'i I 
letter of y is a, and Pcr = -j^ ■ For self- reducible sets | Scr \ can be recursively approximated using the same 

I S' I 

technique. Let p^.a' — 15 | and so on, until one reaches | Sai,...,a^ \ ^i iti = \y\ — 1, which can be directly 
computed. Then 

\S\^ 1 s-i,-,-^ I 



P(Ji -Pa 1. 172 I ■■■TPcri,...,a„ 



Let 'jhj be the estimated measure for p„ obtained with the uniform generator for y. The Pai,....cji can be 
replaced by their estimates and leading to an estimator for | 5 |. 

Conversely, one can reduce approximate uniform generation to approximate counting. Compute | 8^ \ 
and \ S \. Suppose S = {0, 1} and let po = jiy- Generate with probability po £^nd 1 with probability 1 — po 
and recursively apply the same method. If one obtains as the first bit, one sets poo — t§\ and generates 
as the next bit with probability poo and 1 with probability 1 — pooj and so on. One obtains a string y (z S 
with an approximate uniform distribution. 

3.2.3 Learning 

In the general setting, given a black box, i.e. an unknown function /, and samples Xi,yi — f{xi) for 
i = 1, ..., iV, one wishes to find /. Classical learning theory distinguishes between supervised and unsupervised 
learning. In supervised learning / is one function among a class T of given functions. In unsupervised 
learning, one tries to find g as the best possible function. 

Learning models suppose membership queries, i.e. positive and negative examples, i.e. given x, an oracle 
produces f{x) in one step. Some models assume more general queries such as conjecture queries: given an 
hypothesis g, an oracle answers YES if / = g, else produces an x where / and g differ. For example, let / be 
a function S* — > {0, 1} where S is a finite alphabet. It describes a language L = {x G 'S* , f{x) = 1} C E*. 
On the basis of membership and conjecture Queries, one tries to output g = f ■ 

Angluin's Learning algorithm for regular sets The learning model is such that the teacher answers 
membership queries and conjecture queries. Angluin's algorithm shows how to learn any regular set, i.e. 
any function E* — >■ {0, 1}, which is the characteristic function of a regular set. It finds / exactly, and the 
complexity of the procedure depends polynomially 0{m.n'^) on two parameters: n the size of the mini- 
mum automaton for / and m the maximum length of counter examples returned by the conjecture queries. 
Moreover there are at most n conjecture Queries. 

Learning without reset The Angluin model supposes a reset operator, similar to the reliable reset of 
section 



2.3.1 but RS93] showed how to generalize the Angluin model without reset. As seen in Section 



2.3. 1[ a homing sequence is a sequence which uniquely identifies the state after reading the sequence. Every 



minimal deterministic finite automaton has a homing sequence a. 
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The procedure runs n copies of Angluin's algorithm, Li, ...,Ln, where Li assumes that Si is the initial 
state. After a membership query in Li, one applies the homing sequence a, which leads to state Sk- One 
leaves Li and continues in Lk- 

3.3 Methods for approximate decision problems 

In the previous section, we considered approximate methods for decision, counting and learning problems. 
We now relax the decision and learning problems in order to obtain more efficient approximate methods. 

3.3.1 Property testing 

Property testing is a statistics based approximation technique to decide if either an input satisfies a given 
property, or is far from any input satisfying the property, using only few samples of the input and a specific 



distance between inputs. It is later used in section 4.2 The idea of moving the approximation to the input 



was implicit in Program Checking |BK951 lBLR93l IRS96| . in Probabilistically Checkable Proofs (PGP) |AS98| . 
and explicitly studied for graph properties under the context of property testing I GGR98| . The class of 
sublinear algorithms has similar goals: given a massive input, a sublinear algorithm can approximately 
decide a property by sampling a tiny fraction of the input. The design of sublinear algorithms is motivated 
by the recent considerable growth of the size of the data that algorithms are called upon to process in 
everyday real-time applications, for example in bioinformatics for genome decoding or in Web databases for 
the search of documents. Linear-time, even polynomial-time, algorithms were considered to be efficient for 
a long time, but this is no longer the case, as inputs are vastly too large to be read in their entirety. 

Given a distance between objects, an e-tester for a property P accepts all inputs which satisfy the property 
and rejects with high probability all inputs which are e-far from inputs that satisfy the property. Inputs 
which are e-close to the property determine a gray area where no guarantees exists. These restrictions allow 
for sublinear algorithms and even 0(1) time algorithms, whose complexity only depends on e. 

Let K be a class of finite structures with a normalized distance dist between structures, i.e. dist lies in 
[0, 1]. For any e > 0, we say that U,U' G K. are e-close if their distance is at most e. They are e-far if they 
are not e-close. In the classical setting, satisfiability is the decision problem whether U \= P for a structure 
[/ e K and a property P C K. A structure C/ € K e-satisfies P, or U is e-close to K or [/ |=£ P for short, 
if U is e-close to some [/' e K such that U' \= P. We say that U is e-far from K or [/ ^^ P for short, if U 
is not e-close to K. 

Definition 8 (Property tester (GGR98]) Let e > 0. An e-tester for a property P C K is a randomized 
algorithm A such that, for any structure t/ G K as input: 

(1) If U \^ P , then A accepts; 

(2) If U ^e P, then A rejects with probability at least 2/3r| 

A query to an input structure U depends on the model for accessing the structure. For a word w, a query 
asks for the value of w[i], for some i. For a tree T, a query asks for the value of the label of a node i, and 
potentially for the label of its parent and its j'-th successor, for some j. For a graph a query asks if there 
exists an edge between nodes i and j. We also assume that the algorithm may query the input size. The 
query complexity is the number of queries made to the structure. The time complexity is the usual definition, 
where we assume that the following operations are performed in constant time: arithmetic operations, a 
uniform random choice of an integer from any finite range not larger than the input size, and a query to the 
input. 

Definition 9 A property P (- K. is testable, if there exists a randomized algorithm A such that, for every 
real e > as input, A{e) is an e-tester of P whose query and time complexities depend only on e (and not 
on the input size). 



^The constant 2/3 can be replaced by any other constant < 7 < 1 by iterating 0(log(l/7)) the e-tester and accepting iff 
all the executions accept 
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Tools based on property testing use an approximation on inputs which allows to: 

1. Reduce the decision of some global properties to the decision of local properties by sampling, 

2. Compress a structure to a constant size sketch on which a class of properties can be approximated. 
We detail some of the methods on graphs, words and trees. 



Graphs In the context of undirected graphs |GGR98) . the distance is the (normalized) Edit distance on 
edges: the distance between two graphs on n nodes is the minimal number of edge- insertions and edge- 
deletions needed to modify one graph into the other one. Let us consider the adjacency matrix model. 
Therefore, a graph G = (V, E) is said to be e-close to another graph G", if G is at distance at most en^ from 
G", that is if G differs from G' in at most en'^ edges. 

In several cases, the proof of testability of a graph property on the initial graph is based on a reduction 
to a graph property on constant size but random subgraphs. This was generalized for every testable graph 
properties by |GT03j . The notion of e-reducibility highlights this idea. For every graph G = {V,E) and 
integer fc > 1, let 11 denote the set of all subsets tt CV oi size k. Denote by G^ the vertex- induced subgraph 
of G on TT. 

Definition 10 Let e > be a real, fc > 1 an integer, and </>, ip two graph properties. Then (p is (e, fc)-reducible 
to ip if and only if for every graph G, 



Gh 

GV-e 



Vvr e n, G, h V-, 

Pr [G„ ^ V] > 2/3. 

ttGII 



Note that the second implication means that if G is e-far to all graphs satisfying the property 0, then with 
probability at least 2/3 a random subgraph on k vertices does not satisfy ■;/'■ 

Therefore, in order to distinguish between a graph satisfying to another one that is far from all graphs 
satisfying 0, we only have to estimate the probability Pr^gniGir H V']- In the first case, the probability is 1, 
and in the second it is at most 1/3. This proves that the following generic test is an e-tester: 



Generic Test(i/),e, fc) 








1. 


Input: A graph G = 


{V.E) 






2. 


Generate uniformly 


a random subset tt 


C T/of 


size fc 


3. 


Accept if Gtt H V' and reject otherwise 







Proposition 1 If for every e > 0, there exists k^ such that <j) is {e,k^) -reducible to ip, then the property 4> 
is testable. Moreover, for every e > 0, Generic Testis, e,k^) is an e-tester for (p whose query and time 
complexities are in (k^Y . 

In fact, there is a converse of that result, and for instance we can recast the testability of c- 
colorability |GGR98l IAK02J in terms of e-reducibility. Note that this result is quite surprising since c- 
colorability is an NP-complcte problem for c > 3. 

Theorem 4 ( [AK02) ) For all c> 2, e > 0, c-colorability is {e,0{{c\nc)/e'^)) -reducible to c-colorability. 
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Words and trees Property testing of regular languages was first considered in |AKNSOO] for the Ham- 
ming distance, and then extended to languages recognizable by bounded width read-once branching pro- 
grams ^New02i , where the Hamming distance between two words is the minimal number of character substi- 
tutions required to transform one word into the other. The (normalized) edit distance between two words 
(resp. trees) of size n is the minimal number of insertions, deletions and substitutions of a letter (resp. 
node) required to transform one word (resp. tree) into the other, divided by n. When words are infinite, the 
distance is defined as the superior limit of the distance of the respective prefixes. 

|MdR07) considered the testability of regular languages on words and trees under the edit distance with 
moves, that considers one additional operation: moving one arbitrary substring (resp. subtree) to another 
position in one step. This distance seems to be more adapted in the context of property testing, since 
their tester is more efficient and simpler than the one of [AKNSOO] . and can be generalized to tree regular 
languages. 

|FMdR10] developed a statistical embedding of words which has similarities with the Parikh map- 
ping jPar66| . This embedding associate to every word a sketch of constant size (for fixed e) which allows to 
decide any property given by some regular grammar or even some context-free grammar. This embedding 
has other implications that we will develop further in Section [4. 2. 3[ 

3.3.2 PAC and statistical learning 

The Probably Approximately Correct (PAC) learning model, introduced by Valiant jVal84j is a framework to 
approximately learn an unknown function / in a class T, such that each / has a finite representation, i.e. 
a formula which defines /. The model supposes positive and negative samples along a distribution V, i.e. 
values Xi,f{xi) for i = 1,2,. ..,7V. The learning algorithm proposes a function h and the error between / 
and h along the distribution V is: 

error{h) = Pr [f{x) ^ h{xy\ 

A class J^ of functions / is PAC-learnable if there is a randomized algorithm such that for all f E T,e,d,T>, 
it produces with probability greater than 1 — S, an estimate h for / such that error{h) < e. It is efficiently 
PAC-learnable if the algorithm is polynomial in N,-,^,size{f), where size{f) is the size of the finite 
representation of /. Such learning methods are independent of the distribution V, and are used in black box 
checking in section |4.3| to verify a property of a black box by learning a model. 

The class H of the functions h is called the Hypothesis space and the class is properly learnable if H is 
identical to J^: 

• Regular languages are PAC-learnable. Just replace in Angluin's model, the conjecture queries by PAC 
queries, i.e. samples from a distribution V. Given a proposal L' for L, we take N samples along V 
and may obtain a counterexample, i.e. an element x on which L and L' disagree. If n is the minimum 
number of states of the unknown L, then Angluin's algorithm with at most 

N = 0{{n + l/e).(nln(l/(5) + n^) 

samples can replace the n conjecture queries and guarantee with probability at least 1 — d that the 
error is less than e. 

• fc-DNF and /c-CNF are learnable but it is not known whether CNF or DNF are learnable. 

The Vapnik-Chernovenkis (VC) dimension |VC81| of a class J^, denoted VC{J-) is the largest cardinality 
d of a sample set S that is shattered by T, i.e. such that for every subset S' C S there is an / g J^ such 
that f{x) = a for x G 5", f{x) = b iov x € S - S' and a ^ b. 

A classical result of |BEHW89| IKV94J is that if d is finite then the class is PAC learnable. If A^ > 
0{-. log J -I- -.log --^), then any h which is consistent with the samples, i.e. gives the same result as / on 
the random samples, is a good estimate. Statistical learning |Vap83| generalizes this approach from functions 
to distributions. 
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4 Applications to model checking and testing 

4.1 Bounded and unbounded model checking 

Recall that the Model Checking problem is to decide, given a transition system M with an initial state sq 
and a temporal formula ip whether M,so \= f, i-e. if the system M satisfies the property defined by (p. 
Bounded model checking introduced in |BCCZ99b] is a useful method for detecting errors, but incomplete 
in general for efficiency reasons: it may be intractable to ensure that a property is satisfied. For example, 
if we consider some safety property expressed by a formula tp = Gp, A4, sq |= V(^ means that all initialized 
paths in A4 satisfy if, and A4,so \= 3(^ means that there exists an initialized path in A4 which satisfies p. 
Therefore, finding a counterexample to the property Gp corresponds to the question whether there exists a 
path that is a witness for the property F^p. 

The basic idea of bounded model checking is to consider only a finite prefix of a path that may be a 
witness to an existential model checking problem. The length of the prefix is restricted by some bound k. In 
practice, one progressively increases the bound, looking for witnesses in longer and longer execution paths. 
A crucial observation is that, though the prefix of a path is finite, it represents an infinite path if there is a 
back loop to any of the previous states. If there is no such back loop, then the prefix docs not say anything 
about the infinite behavior of the path beyond state s^. 

The /c-bounded semantics of model checking is defined by considering only finite prefixes of a path, with 
length fc, and is an approximation to the unbounded semantics. We will denote satisfaction with respect to 
the fc-bounded semantics by |=fc. The main result of bounded model checking is the following. 

Theorem 5 Let (p be an LTL formula and M be a transition system. Then Ai \= 3p iff there exists 
k = 0(|X|.2l'^l) such that M |=fe 3(^. 

Given a model checking problem A^ |— 3p, a typical application of BMC starts at bound and increments 
the bound k until a witness is found. This represents a partial decision procedure for model checking 
problems: 

• if Al 1= 3(^, a witness of length k exists, and the procedure terminates at length k. 

• if A4 ^ 3(/3, the procedure does not terminate. 

For every finite transition system M and LTL formula (j), there exists a number k such that the absence 
of errors up to k proves that M |= V(/). We call k the completeness treshold of M with respect to (j). For 
example, the completeness treshold for a safety property expressed by a formula Gp is the minimal number 
of steps required to reach all states: it is the longest "shortest path" from an initial state to any reachable 
state. 

In the case of unbounded model checking, one can formulate the check for property satisfaction as a 
SAT problem. A general SAT approach |ABE00b") can be used for reachability analysis, when the binary 
relation R is represented by a Reduced Boolean Circuit (RBC), a specific logical circuit with A,^^^. One 
can associate a SAT formula with the binary relation R and each i?* which defines the states reachable at 
stage i from Sq, i.e. i?" = {sq}, i?*+^ = {s : 3vR^{v) A vRs}. Reachability analysis consists in computing 
unary sets T*, for i — I, ...,m: 

• T' is the set of states reachable at stage i which satisfy a predicate Bad, i.e. 3s{Bad{s) A R'^{s)), 

• compute T*+i and check if T' o T^+i. 

In some cases, one may have a more succinct representation of the transitive closure of R. A SAT solver 
is used to perform all the decisions. 
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4.1.1 Translation of BMC to SAT 

It remains to show how to reduce bounded model checking to prepositional satisfiability. This reduction 
enables to use efficient propositional SAT solvers to perform model checking. Given a transition system 
A4 = (S, I, R, L) where / is the set of initial states, an LTL formula if and a bound k, one can construct a 
propositional formula [M,ip]k such that: 

M \=k 3(/9 iff [M, f]k is satisfiable 

Let (so,...,Sfc) the finite prefix, of length fc, of a path a. Each Si represents a state at time step i 
and consists of an assignment of truth values to the set of state variables. The formula [A^,(^]fc encodes 
constraints on (sq, . . . , Sfe) such that [A^, ip]k is satisfiable iff cr is a witness for ip. 

The first part [A4]k of the translation is a propositional formula that forces (sq, . . . ,Sfc) to be a path 
starting from an initial state: [A4]k = -^ (sq) A Ai=o -^(■^i: Sj+i)- 

The second part [ip\k is a propositional formula which means that a satisfies ip for the fc-bounded seman- 
tics. For example, if tp is the formula Fp, the formula [p\k is simply the formula Vi=oP('^0- ^^ general, the 
second part of the translation depends on the shape of the path a: 

• If (T is a fc-loop, i.e. if there is a transition from state Sk to a state si with I < k, we can define a 
formula [lysjfc,;, by induction on p, such that the formula Vi=o(^('^fe' ^0 ^ Iv'l's,') means that a satisfies 
ip. 

• If (T is not a fc-loop, we can define a formula [v?]fe, by induction on p, such that the formula 
(^ V;=o R{^k,si)) A [p]k means that a satisfies (p for the fc-bounded semantics. 

We now explain how interpolation can be used to improve the efficiency of SAT based bounded model 
checking. 

4.1.2 Interpolation in propositional logic 

Craig's interpolation theorem is a fundamental result of mathematical logic. For propositional formulas A 
and B, a A ^ B, there is a formula A' in the common language of A, B such that A ^ A! and A! — > B. 
Example: A = p A q, B = qW r. Then A' = q. 

In the model checking context, |McM03| proposed to use the interpolation as follows. Consider formulas 
A, B in CNF normal form, and let {A, B) be the set of clauses of A and B. Instead of showing A — > C, we 
set B = -^C and show that {A, B) is unsatisfiable. 

If {A, B) is unsatisfiable, we apply Craig's theorem and conclude that there is an A' such that A -^ A' 
and (A', B) is unsatisfiable. Suppose A is the set of clauses associated with an automaton or a transition 
system and B is the set of clauses associated with the negation of the formula to be checked. Then A' defines 
the possible errors. 

There is a direct connection between a resolution proof of the unsatisfiability of {A, B) and the interpolant 
A' . It suffices to keep the same structure of the resolution proof and only modify the labels, as explained in 
Figure [6] 

Resolution rule. Given two clauses Ci,C2 such that a variable p appears positively in Ci and negatively 
in C2, i.e. Ci = pM C'l and C2 = ~'P V C2, the resolution rule on the pivot p yields the resolvent C = C^ V C2. 
If the two clauses are Ci = p and C2 = —'P, the resolvent on pivot p is _L (the symbol for false). The proof 11 
of unsatisfiability of {A, B) by resolution, can be represented by a Directed Acyclic Graph DAG with labels 
on the nodes, several input nodes and one output node, as in figure l6|(I). Clauses of A, B are labels of the 
input nodes, clauses C obtained by one application of the resolution rule are labels of the internal nodes, 
and _L is the label of the unique output nodes. 

Obtaining an interpolant. For the sets of clauses (A, _B), a variable v is global if it occurs both in 
A and in B, otherwise it is local to A or to B. Let g be a function which transforms a clause into another 
clause. For a clause C £ A, let g{C) be the disjunction of its global litterals, let g{C) ~ _L (false) if no global 
litteral is present and let g{C) = T (true) if C G B. 
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Clauses; (p) (-p +q) (-1+'') (-r) 



(p) (-P +q) 



(q) (-q+r) 



(-r) (r) 



q 

a) (II) 



Figure 6: Craig Interpolant: A : {{p), {^p V q)}, and B : {{^q V r), (^r)}. The proof by resolution (I) shows 
that {A, B) is unsatisfiable. The circuit (II) (with OR and AND gates, input labels which depend on the 
clauses, as explained in definition llll) mimics the proof by resolution and output the interpolant A' — q. 



The labels of the internal nodes and output node are specified by definition II on a copy 11' of 11. 
Definition 11 For all labels C of nodes of II, let fi{C) be a boolean formula which is the new label ofH'. 

• if C is the label of an input node then fJ.{C) — g{C). 

• let C be a resolvent on Ci,C2 using the pivot p: if p is local to A, then p{C) = p{Ci) V p{C2) otherwise 
p{C)^piCi)Ap{C2) 

The interpolant of (A, B) along 11 is p-{-L), i.e. the clause associated with the DAG's unique output node. 

This construction yields a direct method to obtain an interpolant from an an unsatisfiability proof. It isolates 
a subset of the clauses from A,B, which can be viewed as an abstraction of the unsatisfiability proof. This 
approach is developped further in |HJMM04] . 

4.1.3 Interpolation and SAT based model checking 

One can formulate the problem of safety property verification in the following terms |McM03j . Let Ai = 
{S,R,I,L) be a transition system and F a final constraint. The initial constraint /, the final constraint 
F and the transition relation R are expressed by propositional formulas over boolean variables (a state is 
represented by a truth assignment for n variables (wi, . . . ,««))■ 

An accepting path of Al is a sequence of states {sq, . . . ,Sk) such that the formula /(sq) A 
(Ai=o ^(*ii Si+i)) A F{sk) is true. In bounded model checking, one translates the existence of an accepting 
path of length < i < k + I into a propositional satisfiability problem by introducing a new indexed set of 
variables Wi = {wn, . . . , Win} ioi < i < k + 1. An accepting path of length in the range {0, . . . , fc + 1} 
exists exactly when the following formula is satisfiable: 

k fe + l 



6mcS = I{Wo) A (/\ RiW,,W,+,)) A ( \/ F(W^,)) 



In order to apply the interpolation technique, one expresses the existence of a prefix of length I and of a 
suffix of length k by the following formulas: 

pre^iM) ^ I{Wo) A RiWo,Wi) 
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k k+1 

sufliM) = (/\ R{W,,W,+i)) A ( V F{W,)) 

i=l i=l 

To apply a SAT solver, one assumes the existence of some function CNF that translates a boolean 
formula / into a set of clauses CNF{f, U) where U is a set of fresh variables, not occurring in /. 
Given two sets of clauses A, B such that A U B is unsatisfiable and a proof 11 of unsatisfiability, we 
note Interpolantili, A, B) the associated interpolant. Below, we give a procedure to check the existence 
of a finite accepting path of A^, introduced in |McM03j . The procedure is parametrized by a fixed value fc > 0. 

Procedure FiniteRun(M = (/, R, F),k) 

if {I A F) is satisGable, return True 

letT^I 

while (true) 

let M' = iT,R,F), A = CiVF(prei(M'), C/i), B = CNF{sufl{M'),U2) 

Run SAT onAuB 

If (AU B is satisfiable) then 

ifT = I then return True else abort 
else (if AU B unsatisfiable) 

let n be a proof of unsatisfiability ofAUB,P^ Interpolant(n, A, B), T' = P(W/Wo) 

ifT' implies T return False 

letT ^T\JT' 
endwhile 
end 

Theorem 6 ( ]McMO^ ) For k > 0, if FiniteRun(A4,k) terminates without aborting, it returns True iff M 
has an accepting path. 

This procedure terminates for sufficiently large values of k: the reverse depth of Ai is the maximum 
length of the shortest path from any state to a state satisfying F. When the procedure aborts, one only has 
to increase the value of k. Eventually the procedure will terminate. Using interpolation in SAT based model 
checking is a way to complete and accelerate bounded model checking. 

4.2 Approximate model checking 

We first consider a heuristics (Monte-Carlo) to verify an LTL formula, and then consider two methods where 
both approximation and randomness are used to obtain probabilistic abstractions, based on property and 
equivalence testers. 

4.2.1 Monte-Carlo model checking 

In this section, we present a randomized Monte-Carlo algorithm for linear temporal logic model checking 
[GS05| . Given a deterministic transition system Jv[ and a temporal logic formula 0, the model checking 
problem is to decide whether M. satisfies 0. In case is linear temporal logic (LTL) formula, the problem 
can be solved by reducing it to the language emptiness problem for finite automata over infinite words 
[VW86J . The reduction involves modeling M. and ^0 as Biichi automata Am and A-,^, taking the product 
A — Am X A-,ij,, and checking whether the language L{A) of A is empty. Each LTL formula can be 
translated to a Biichi automaton whose language is the set of infinite words satisfying (p by using a tableau 
construction. 

The presence in A of an accepting lasso, where a lasso is a cycle reachable from an initial state of A^ means 
that M is not a model of 6. 
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Estimation method. To each instance Ai \^ cj) oi the LTL model checking problem, one may associate 
a Bcrnouilli random variable z that takes value 1 with probability pz and value with probability 1 — pz- 
Intuitively, pz is the probability that an arbitrary execution path of A^ is a counterexample to 4>. Since pz 
is hard to compute, one can use a Monte-Carlo method to derive a one-sided error randomized algorithm for 
LTL model checking. 

Given a Bcrnouilli random variable Z, define the geometric random variable X with parameter pz whose 
value is the number of independent trials required until success. The probability distribution of X is: 

p{N) = Pr[X = N] = q^-\pz 

where Qz — I ^ Pz, and the cumulative distribution is 

N 
ri=0 

Requiring that Pr[X < N] = 1 — S ior confidence ratio 6 yields: N = ln{5)/ln{\ — pz) which provides the 

number of attempts N needed to achieve success with probability greater 1 — 5. Given an error margin e 

and assuming the hypothesis pz > e, we obtain that: 

M = ln{5)/ln{l - e) > ln{5) / ln{l - p z) and Pr[X <M]> Pr[X <N]>l-5. 

Thus M is the minimal number of attempts needed to achieve success with confidence ratio 5, under the 

assumption pz > £. 

Monte-Carlo algorithm. The MC algorithm samples lassos in the automaton A via a random walk 
through A's transition graph, starting from a randomly selected initial state of A^ and decides if the cycle 
contains an accepting state. 

Definition 12 A finite run a ~ s^^x^SiXi . . . s„a;„s„+i of a Bilchi automaton A = (S, 5*, Sg, R, F) is called 
a lasso if Sf), . . . , s„ are pairwise distinct and Sn+i — Si for some < i < n. Moreover, a is said an accepting 
lasso if some Sj (z F (i < j < n), otherwise it is a non accepting lasso. The lasso sample .space L of A 
is the set of all lassos of A, while La and L„ are the sets of all accepting and non accepting lassos of A, 
respectively. 

To obtain a probability space over L, we define the probability of a lasso. 

Definition 13 The probability Pr[a] of a finite run a — sqXq . . . Sn-iXnSn of a Biichi automa- 
ton A is defined inductively as follows: Pr[so] ~ 1/fc if |so| — k and Pr[sQXoXi . . . Sn-iXnSn] = 
Pr[soXo . ■ . Sn-i].Tr{sn-i,XnjSn) whcre i:{s,x,t) — 1/m if {s,x,t) G R and \R{s)\ = m. Recall that 
R{s) = {t:3xeT,, (s, x, t) e R}. 

Note that the above definition explores uniformly outgoing transitions and corresponds to a random walk 
on the probabilistic space of lassos. 

Proposition 2 Given a Bilchi automaton A, the pair {V{L),Pr) defines a discrete probability space. 

Definition 14 The random variable Z associated with the probability space 'P{L),Pr is defined by: pz = 
Pr[Z = 1] = E.eL„ P^ «^rf qz = Pr[Z = 0] = E.eL„ PM- 

Theorem 7 Given a Bilchi autom,aton A and parameters e and 5 if MC returns false, then L{A) ^ 0. 
Otherwise, Pr[X > M\Ho] < 5 where M = ln{5)/ln{\ - e) and Hq = pz > e. 

This Monte-Carlo decision procedure has time complexity O(M.D) and space complexity 0(D), where D 
is the diameter of the Biichi product automaton. 

This approach by statistical hypothesis testing for classical LTL model checking has an important draw- 
back: liQ < Pz < e, there is no guarantee to find a corresponding error trace. However, it would be possible 
to improve the quality of the result of the random walk by randomly reinitializing the origin of each random 
path in the connected component of the initial state. 
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4.2.2 Probabilistic abstraction 

Symbolic model checking |McM93[ ICGP99| uses a succinct representation of a transition system, such as 
an ordered binary decision diagrams (OBDD) |Bry86[ |Bry91| or a SAT instance. In some cases, such as 
programs for integer multiplication or bipartiteness, the OBDD size remains exponential. The abstraction 



method (see Section 3.2.11 provides a solution in some cases, when the OBDD size is intractable. We now 
consider random substructures (A^)^ of finite size, where tt denotes the random parameter, and study cases 
when we can infer a specification SPEC in an approximate way, by checking whether random abstractions 
TT satisfy with sufficiently good probability (say 1/2) on the choice of it, another specification SPEC which 
depends on SPEC and tt. 

We have seen in section |3.3.1| on property testing, that many graph properties on large graphs are e- 
reducible to other graph properties on a random subgraph of constant size. Recall that a graph property (j) 
is e-reducihle to tp if testing ^ on random subgraphs of constant size suffices to distinguish between graphs 
which satisfy (/>, and those that are e-far from satisfying 0. Based on those results, one can define the 
concept of probabilistic abstraction for transition systems of deterministic programs whose purpose is to 
decide some graph property. Following this approach, |LLM"'"07| extended the range of abstractions to 
programs for a large family of graphs properties using randomized methods. A probabilistic abstraction 
associates small random transition systems, to a program and to a property. One can then distinguish with 
sufficient confidence between programs that accept only graphs that satisfy and those which accept some 
graph that is e-far from any graph that satisfies (j). 

In particular, the abstraction method has been applied to a program for graph bipartiteness. On the 
one hand, a probabilistic abstraction on a specific program for testing bipartiteness and other temporal 
properties has been constructed such that the related transition systems have constant size. On the other 
hand, an abstraction was shown to be necessary, in the sense that the relaxation of the test alone does not 
yield OBDDs small enough to use the standard model checking method. To illustrate the method, consider 
the following specification, where </> is a graph property, 

SPEC: The program P accepts only if the graph G satisfies 4>. 

The graph G is described by some input variables of P providing the values of the adjacency matrix of G. 
We consider a transition system M. which represents P, parametrized by the graph input G. The method 
remains valid for the more general specifications, where O is in 3CTL*, 

SPEC: X, G h only if G satisfies cj). 

The formula 0, written in temporal logic, states that the program reaches an accepting state, on input 
G. The states of M are determined by the variables and the constants of P. The probabilistic abstraction 
is based on property testing. Fix k an integer, e > a real, and another graph property ip such that is 
(e, fc)-reducible to ip. Let 11 be the collection of all vertex subsets of size k. The probabilistic abstraction is 
defined for any random choice of tt e 11. For all vertex subsets tt € 11, consider any abstraction 7W^ for the 
transition system A4 such that the graph G is abstracted to its restriction on tt, that we denote by Gj^. The 
abstraction of the formula Q is done according to the transformation V, defined in Section [3.2.11 

We now present the generic probabilistic tester based on the above abstraction. 



Graph Test{{n,M),Q,'ip) 






1. Randomly choose a vertex subset tt e 11. 






2. Accept iff VG^ {M" \- V{e) =^ 


G. 


= V')- 



The following theorem states the validity of the abstraction. 

Theorem 8 Let be in 3CTL*. Let e > be a real, k > 1 an integer, and (j) be a formula (e, k)-reducible 
to ip. If there exists a graph G such that M,G \= and G |^e (p, then Graph Test( (II, Ai),Q,'4') rejects 
with probability at least 2/3. 
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This approximate method has a time complexity independent of n, the size of the structure, and only 
dependent on e. 

4.2.3 Approximate abstraction 

In |FMdR10] . an equivalence tester is introduced and decides if two properties are identical or e-far, i.e. if 
there is a structure which satisfies one property but which is e-far from the other property, in time which 
only depends on e. It generalizes property testing to Equivalence Testing in the case we want to distinguish 
two properties, and has direct applications for approximate model checking. 

Two automata defining respectively two languages Li and L2 are e-equivalent when all but finitely 
many words w ^ Li are e-close to L2, and conversely. The tester transform both transition systems and 
a specification (formula) into Biichi automata, and test their approximate equivalence efficiently. In fact, 
the e-equivalence of nondeterministic finite automata can be decided in deterministic polynomial time, that 
is to'^I " whereas the exact decision version of this problem is PSPACE-complete by |,SM73j . and in 
deterministic exponential time algorithm for the e-equivalence testing of context-free grammars, whereas the 
exact decision version is not recursively computable. 

The comparison of two Biichi automata is realized by computing a constant size sketch for each of them. 
The comparison is done directly on the sketches. Therefore sketches are abstractions of the initial transition 
systems where equivalence and implication can be approximately decided. More precisely, the sketch is an 
£i-embedding of the language. Fix a Biichi automaton A. Consider all the (finite) loops of A that contains 
an accepting state, and compute the statistics of their subwords of length I/e. The embedding 'H(/l) is 
simply the set of these statistics. The main result states that approximate equivalence on Biichi automata 
is characterized by the ^i-embedding in terms of statistics of their loops. 

Theorem 9 Let A, B be two Biichi automata. If A and B recognize the same language then H^A) — 'H(B). 
If A (respectively B) recognizes an infinite word w such that B (respectively A) does not recognize any word 
e/A-close to w, then UiA) ^ HiB). 

This approximate method has a time complexity polynomial in the size of the automata. 

4.3 Approximate black box checking 

Given a black box A, a Conformance test compares the black box to a model B for for a given conformance 



relation (cf Section 2.3.21, whereas Black Box Checking verifies if the black box A satisfies a property 
defined by a formula ip. When the conformance relation is the equivalence, conformance testing can use 
the Vasilevskii-Chow method |Vas73| . which remains an exponential method 0{P.n.p'^~''~^^), where I is the 
known number of states of the model -B, and n is a known upper-bound for \A\ {n > I) and p is the size of 
the alphabet. 

4.3.1 Heuristics for black box checking 

[PVY99| proposes the following 0(p") strategy to check if a black box A satisfies a property tp. They build 
a sequence of automata Mi,M2, ...,Mi, ... which converges to a model B of A, refining Angluin's learning 
algorithm. The automaton Mi is considered as a classical automaton and as a Biichi automaton which 
accepts infinite words. Let P be a Biichi automaton, introduced in section [2.1. 1 [ associated with ^ip. Given 
two Biichi automata, P and Mj, one can use model checking to test if the intersection is empty, i.e. if 
L{Mi) n L{P) = 0: this operation is exponential in the size of the automata. 

If L(Mi) n L{P) ^ 0, there is ui, cr2 such that ci.cr^ is in Mi as a Biichi automaton and in P, and such 
that CTi.CTj^^ is accepted by the classical Mi. Apply ai.a!^^^ to A. If A accepts there is an error as A also 
accepts cTi.cr^, i.e. an input which does not satisfy the property. If A rejects then M, and A differ and one 
can use Angluin's algorithm to learn Mi^i from Mi and the separating sequence a = tri.cTg ■ 

If L{Mi) n L{P) — 0, one can compare Mi with A using Vasilevskii-Chow's conformance algorithm. 
If they are different, the algorithm provides a sequence a where they differ and one can use the learning 
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algorithm to propose M^+i with more states. If the conformance test succeeds and k = \Mi\, one keeps 
applying it with larger values of k, i.e. k + 1, ...,ti. See Figure[7| The pseudo-code of the procedure is: 



Black box checking strategy {A,P,n). 

• Set L{Mi) = 0. 

• Loop: L{Mi) n L{P) 7^ ? (model checking). 



— If L(Mi) n L{P) 7^ 0, the intersection contains some ai.a^ such that (Ji.a-2 € L{Mi) for all finite 
j. Enter Wi — reset.ai.a^ to A. If A accepts then there is an error as there is a word in 
L{P) n L{A), then Reject. If A rejects then A 7^ Mi, then go to Learn Mi+i{wi). 

— li L{Mi) r\ L{P) =0. 

Conformance: check whether Mi of size k conforms with A with the Vasilevskii-Chow algorithm 
with input A, Mi, k. If not, Vasilevskii-Chow provides a separating sequence tr, then go to Learn 
Mj_|_i(i7). li k — n then Accept, else set k — k + 1 and go to Conformance. 

— Learn Mi^i{a): Apply Angluin algorithm from Mi and the sequence a not in Mi. Go to Loop. 

This procedure uses model checking, conformance testing and learning. If one knows B, one could 
directly use the Vasilevskii-Chow algorithm with input A,B,n but it is exponential, i.e. 0(p"^'+^). With 
this strategy, one tries to discover errors by approximating A with Mi with k states and hopes to catch 
errors earlier on. The model checking step is exponential and the conformance testing is only exponential 
when k > L 

We could relax the black box checking, and consider close inputs, i.e. decide if an input x accepted by A 
is e close to tp and hope for a polynomial algorithm in n. 

Ml 

t 

MODEL CHECKER 
Mi , P 





(b) Mi converge to B 



(a) Learning iterations 

Figure 7: Peled-Vardi-Yanakakis learning Scheme in (a), and the sequence of M^ in (b). 



4.3.2 Approximate black box checking for close inputs 

In the previous Figure [7J we can relax the model checking step, exponential in n, by the approximate model 
checking, polynomial in n, as in section 4.2 Similarly, the conformance equivalence could be replaced by 
an approximate version where we consider close inputs, i.e. inputs with an edit distance with moves less 
than e. In this setting. Approximate Conformance checks whether Mi of size k conforms within e with A. 
It is an open problem whether there exists a randomized algorithm, polynomial time in n, for Approximate 
Conformance Testing. 
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4.4 Approximate model-based testing 

In this subsection we first briefly present a class of metfiods tliat are, in some sense, dual to the previous 
ones: observations from tests are used to learn partial models of components under tests, from which further 
tests can be derived. Then we present an approach to random testing that is based on uniform generation 
and counting seen in Section |3.2.2[ It makes possible to define a notion of approximation of test coverage 
and to assess the results of a random test suite for such approximations. 

4.4.1 Testing as learning partial models 

Similarities between testing and symbolic learning methods have been noticed since the early eighties |BA82[ 
ICS87| . Recently, this close relationship has been formalized by Berg et al. in BGJ+05) . However, the few 



reported attempts of using Angluin's-likc inference algorithms for testing have been faced to the difficulty 
of implementing an oracle for the conjecture queries. Besides, Angluin's algorithm and its variants are 
limited to the learning of regular sets: the underlying models are finite automata that are not well suited 
for modeling software. 

|SLG07) propose a testing method where model inference is used for black box software components, 
combining unit testing (i.e. independent testing of each component) and integration testing (i. e. global 
testing of the combined components). The inferred models are PFSM (Parametrized FSM), that are the 



following restriction of EFSMs (cf. Section 2.3.3): inputs and outputs can be parametrized by variables, but 
not the states; transitions are labelled by some parametrized input, some guard on these parameters, and 
some function that computes the output corresponding to the input parameters. 

The method alternates phases of model inference for each components, that follow rather closely the 
construction of a conjecture in Angluin's algorithms, and phases of model-based testing, where the model 
is the composition of the inferred models, and the lUT is the composition of the components. If a fault 
is discovered during this phase, it is used as a counter-example of a conjecture query, and a new inference 
phase is started. 

There are still open issues with this method. It terminates when a model-based testing phase has found 
no fault after achieving a given coverage criteria of the current combined model: thus, there is no assessment 
of the approximation reached by the inferred models, which is dependent of the choice of the criteria, and 
there is no guarantee of termination. Moreover, performing model-based testing on such global models may 
lead to state explosion, and may be beyond the current state of the art. 

4.4.2 Coverage-biased random testing 

In presence of very large models, drawing at random checking sequences is one of the practical alternatives 



to their systematic and exhaustive construction, as presented in Section 2.3.1 

Testing methods based on random walks have already been mentioned in Section |2.3.4| However, as 
noted in |SG03j . classical random walk methods have some drawbacks. In case of irregular topology of the 
underlying transition graph, uniform choice of the next state is far from being optimal from a coverage point 
of view (see Figure Isl). Moreover, for the same reason, it is generally not possible to get any estimation of 
the test coverage obtained after one or several random walks: it would require some complex global analysis 
of the topology of the model. 

One way to overcome these problems has been proposed for program testing in [CDGMOT] IDGG04[ 
|DGG+12| . and is applicable to model-based testing. It relies upon techniques for counting and drawing 
uniformly at random combinatorial structures seen in Section [3. 2. 2[ 

The idea of |GDGM01[ IDGG04[ |DGG+12| is to give up, in the random walk, the uniform choice of 
the next state and to bias this choice according to the number of elements (traces, or states, or transitions) 
reachable via each successor. The estimation of the number of traces ensures a uniform probability on traces. 
Similarly by considering states or transitions, it is possible to maximize the minimum probability to reach 
such an element. Counting the traces starting from a given state, or those traces traversing specific elements 



can be efficiently performed with the methods of Section 3.2.2 
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Figure 8: Irregular topology for which classical random walks is not uniform. 



Let D be some description of a system under test. D may be a model or a program, depending on the 
kind of test we are interested in (black box or structural). We assume that D is based on a graph (or a 
tree, or more generally, on some kind of combinatorial structure). On the basis of this graph, it is possible 
to define coverage criteria: all-vertices, all-edges, all-paths-of a certain-kind, etc. More precisely, a coverage 
criterion C characterizes for a given description D a set of elements Ec{D) of the underlying graph (noted 
E in the sequel when C and D are obvious). In the case of deterministic testing, the criterion is satisfied by 
a test suite if every element of the Ec{D) set is reached by at least one test. 

In the case of random testing, the notion of coverage must be revisited. There is some distribution fi 
that is used to draw tests (either input sequences or traces). Given 57, the satisfaction of a coverage criteria 
C by a testing method for a description D is characterized by the minimal probability qc^N{D) of covering 
any element of Ec{D) when drawing N tests. In }TF89| . Thevenod- Fosse and Waeselink called qc'.N{D) the 
test quality of the method with respect to C 

Let us first consider a method based on drawing at random paths in a finite subset of them (for instance 
V<n, the set of paths of length less or equal to n), and on the coverage criteria C defined by this subset. As 
soon as the test experiments are independent, this test quality qc.N{D) can be easily stated provided that 
Qc,i{D) is known. Indeed, one gets qc,N{D) = 1 — (1 — qcA{D))^ . 

The assessment of test quality is more complicated in general. Let us now consider more practicable 
coverage criteria, such as "all- vertices" or "all-edges", and some given random testing method. Uniform 
generation of paths does not ensure optimal quality when the elements of Ec{D) are not paths, but are 
constitutive elements of the graph as, for example, vertices, edges, or cycles. The elements to be covered 
generally have different probabilities to be reached by a test: some of them are covered by all the tests, some 
of them may have a very weak probability, due to the structure of the behavioral graph or to some specificity 
of the testing method. 

Let Ec{D) = {ei, 62, ..., e„i} and for any i € {1, ..., m},pi the probability for the element e^ to be exercised 
during the execution of a test generated by the considered testing method. Let Pmin — min{pi\i g {!,..., rn}}. 
Then 

qC.N{D)>l^{l^Prmnf (1) 

Consequently, the number N of tests required to reach a given quality qc{D) is 



N> 



log{l-qc{D)) 



log{l - Pmin) 

By definition of the test quality, Pmin is just qc.i{D). Thus, from the formula above one immediately 
deduces that for any given Z?, for any given iV, maximizing the quality of a random testing method with 
respect to a coverage criteria C reduces to maximizing qc^i{D), i. e. Pmin- In the case of random testing 
based on a distribution J7, Pmin characterizes, for a given coverage criteria C, the approximation of the 
coverage induced by ft. 
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However, maximizing Pmin should not lead to give up the randomness of the method. This may be the 
case when there exists a path traversing all the elements of Ec{D): one can maximize p„im by giving a 
probability 1 to this path, going back to a deterministic testing method. Thus, another requirement must be 
combined to the maximization oi pmin- all the paths traversing an element of Ec{D) must have a non null 
probability and the minimal probability of such a path must be as high as possible. Unfortunately, these 
two requirements are antagonistic in many cases. 

In |GDGMdn IDGG04[ IDGG"*" 12] . the authors propose a practical solution in two steps: 

1. pick at random an element e of Ec{D), according to a suitable probability distribution (which is 
discussed below); 

2. generate uniformly at random a path of length < n that goes through e. (This ensures a balanced 
coverage of the set of paths which cover e.) 

Let TTi the probability of choosing element e^ in step 1 of the process above. 

Given ai the number of paths of 7'<„, which cover element e^, given a^j the number of paths, which 
cover both elements e^ and e^ ; (note that ai^i = at and a^j- = a^.i), the probability of reaching e^ by drawing 
a random path which goes through another element Cj is ^^. Thus the probability pi for the element e^ 
(for any i in (l..m)) to be reached by a path is 



Pi 



je(l..m)-{i} 



a," 



J 

3 



The above equation simplifies to 






since a^.i — ai. Note that coefficients aj and a^j are easily computed by ways given in Section 3.2.2 
The determination of the probabilities {tti, 7r2, . . . ,7rm} with ^tt^ — 1, which maximize p, 
min{pi,i S {1, ...,to}} can be stated as a linear programming problem: 

Maximize p„iin under the constraints: < ~ ' '"'" ~ ' ' _, 

[ TTi + 7r2 H (- tt™ = 1 ; 

where the piS are computed as in Equation (pi). Standard methods lead to a solution in time polynomial 
according to m. 

Starting with the principle of a two-step drawing strategy, first an element in Ec{D), second a path 
among those traversing this element, this approach ensures a maximal minimum probability of reaching the 
elements to be covered and, once this element is chosen, a uniform coverage of the paths traversing this 
element. For a given number of tests, it makes it possible to assess the approximation of the coverage, 
and conversely, for a required approximation, it gives a lower bound of the number of tests to reach this 
approximation. 

The idea of biasing randomized test methods in function of a coverage criterion was first studied in the 
nineties in |TFW91| , but the difficulties of automating the proposed methods prevented their exploitation. 
More recently, this idea has been explored also in the Pathcrawler and Dart tools [WMMROSl IGKSQ5J , with 
a limitation to coverage criteria based on paths. 

4.5 Approximate probabilistic model checking 

The main approaches to reduce the prohibitive space cost of probabilistic model checking try to generalize 
predicate abstraction coupled with counterexample-guided abstraction refinement (CEGAR) to a proba- 
bilistic setting. An approach to develop probabilistic CEGAR [HWZOSj is based on the notion and the 
interpretation of counterexamples in the probabilistic framework. A quantitative analog of the well-known 
CEGAR loop is presented in |KKNP09] . The underlying theory is based on representing abstractions of 
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Markov Decision Processes as two-player stochastic games. The main drawback of these approaches is that 
the abstraction step that is repeated during the abstraction refinement process does not ensure a significant 
gain, i.e. exponential, in terms of space. We present now an other approximation method for model checking 
probabilistic transition systems. This approach uses only a succinct representation of the model to check, i.e. 
a program describing the probabilistic transition system in some input language of the model checker. Given 
some probabilistic transition system and some linear temporal formula ip, the objective is to approximate 
Prob[tp] by using probabilistic algoritms whose complexity is logspace. There are serious complexity reasons 
to think that one cannot efiiciently approximate this probability for a general LTL formula. However, if the 
problem is restricted to an LTL fragment sufficient to express interesting properties such than reachability 
and safety, one can obtain efficient approximation algorithms. 

4.5.1 Probability problems and approximation 

The class #P captures the problems of counting the numbers of solutions to NP problems. The counting 
versions of all known A'^P-complete problems arc ^P-complete. The well adapted notion of reduction 
is parsimonious reduction; it is a polynomial time reduction from the first problem to the second one, 
recovering via some oracle, the number of solutions for the first problem from the number of solutions for 
the second one. Randomized versions of approximation algorithms exist for problems such as counting 
the number of valuations satisfying a prepositional disjunctive normal form formula {^DNF) [KLM89' or 
network reliability problem |Kar95j . But we remark that it does not imply the existence of FPRAS for any 
A^P-complete problem. 

A probability problem is defined by giving as input a representation of a probabilistic system and a 
property, as output the probability measure fi of the measurable set of execution paths satisfying this prop- 
erty. One can adapt the notion of fully polynomial randomized approximation scheme, with multiplicative 
or additive error, to probability problems. In the following theorem, RP is the class of decision problems 
that admit one-sided error polynomial time randomized algorithms. 

Theorem 10 There is no fully polynomial randomized approximation scheme (FPRAS) for the problem of 
computing Problil)] for LTL formula ip, unless RP = NP. 

In the following, we give some idea of the proof. We consider the fragment L{F) of LTL in which F is 
the only temporal operator. The following result is due to Clarke and Sistla (SC85| : the problem of deciding 
the existence of some path satisfying a L(F) formula in a transition system is A'^P-complete. Their proof 
uses a polynomial time reduction of SAT to the problem of deciding satisfaction of L(F) formulas. From 
this reduction, we can obtain a one to one, and therefore parsimonious, reduction between the counting 
version of SAT, denoted by ^ffSAT, and counting finite paths, of given length, whose extensions satisfy the 
associated L{F) formula. 

A consequence of this result is the ^^P-hardness of computing satisfaction probabilities for general LTL 
formulas. We remark that if there was a FPRAS for approximating Prob[^p] for LTL formula (f), we could 
efficiently approximate 4t^SAT. A polynomial randomized approximation scheme for ^SAT could be used 
to distinguish, for input y, between the case #(?/) — and the case #(y) > 0, thereby implying a randomized 
polynomial time algorithm for the decision version SAT. 

As a consequence of a result of MRJV86J and a remark of |Sin92) . the existence of an FPRAS for ^SAT 
would imply RP = NP. On the other hand, ^SAT can be approximated with an additive error by a fully 
polynomial time randomized algorithm. In the next section, we determine some restriction on the class of 
linear temporal formulas ip, on the value p = Prob['ip] and only consider approximation with additive error 
in order to obtain efficient randomized approximation schemes for such probabilities. 

4.5.2 A positive fragment of LTL 

For many natural properties, satisfaction on a path of length k implies satisfaction by any extension of this 
path. Such properties are called monotone. Another important class of properties, namely safety properties, 
can be expressed as negation of monotone properties. One can reduce the computation of satisfaction 
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probability of a safety property to the same problem for its negation, that is a monotone property. Let 
consider a subset of LTL formulas which allows to express only monotone properties and for which one can 
approximate satisfaction probabilities. 

Definition 15 The essentially positive fragment (EPF) of LTL is the set of formulas constructed from 
atom,ic formulas (p) and their negations (^p), closed under V, A and the temporal operators X,U. 

For example, formula Fp, that expresses a reachability property, is an EPF formula. Formula Gp, that 
expresses a safety property, is equivalent to ^F-i;?, which is the negation of an EPF formula. Formula 
G{p — ?► Fg), that expresses a liveness property, is not an EPF formula, nor equivalent to the negation of 
an EPF formula. In order to approximate the satisfaction probability Proh[ili\ of an EPF formula, let first 
consider Probk [ip] , the probability measure associated to the probabilistic space of execution paths of finite 
length k. The monotonicity of the property defined by an EPF formula gives the following result. 

Proposition 3 Let ip he an LTL formula of the essentially positive fragment and Ai he a prohabilistic 
transition system. Then the sequence {Probk[i'])keN converges to Prob[ip]. 

A first idea is to approximate Probk[ip] and to use a fixed point algorithm to obtain an approximation 
of Problip]. This approximation problem is believed to be intractable for deterministic algorithms. In the 
next section, we give a randomized approximation algorithm whose running time is polynomial in the size 
of a succinct representation of the system and of the formula. Then we deduce a randomized approximation 
algorithm to compute Problip], whose space complexity is logspace. 

4.5.3 Randomized approximation schemes 

Randomized approximation scheme with additive error. We show that one can approximate the 
satisfaction probability of an EPF formula with a simple randomized algorithm. In practice randomized 
approximation with additive error is sufficient and gives simple algorithms, we first explain how to design 
it. Moreover, this randomized approximation is fully polynomial for bounded properties. Then we will 
use the estimator theorem |KLM89) and an optimal approximation algorithm [DKLROO] in order to obtain 
randomized approximation schemes with multiplicative error parameter, according to definition [5] In this 
case the randomized approximation is not fully polynomial even for bounded properties. 

One generates random paths in the probabilistic space underlying the Kripke structure of depth k and 
computes a random variable A which additively approximates Probk [tp] ■ This approximation will be correct 
with confidence (1 — (5) after a polynomial number of samples. The main advantage of the method is that 
one can proceed with just a succinct representation of the transition system, that is a succinct description 
in the input language of a probabilistic model checker as PRISM. 

Definition 16 A succinct representation, or diagram, of a PTS A4 — {S, Sq, M, L) is a representation of 
the PTS, that allows to generate for any state s, a successor of s with respect to the probability distribution 
induced by M . 

The size of such a succinct representation is substantially smaller than the size of the corresponding 
PTS. Typically, the size of the diagram is polylogarithmic in the size of the PTS, thus eliminating the 
space complexity problem due to the state space explosion phenomenon. The following function Random 
Path uses such a succinct representation to generate a random path of length k, according to the probability 
matrix P, and to check the formula Tp: 



Random Path 

Input: diagramji4 , fc, "0 

Output: samples a path vr of length k and check formula ijj on 

1. Generate a random path tt of length k (with the diagram) 

2. If tp is true on tt then return 1 else 
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Consider now the random sampling algorithm QAA designed for the approximate computation of 

Probk[ip]: 



Generic approximation algorithm QAA 

Input : diagraniM ,k,ij},e,5 

Output: approximation of Prohi^lij}] 

N:=\n{l)/2e^ 

A:=0 

For i = 1 to A^ do ^ := A + Random 'Path.{diagram,M,k,'4}) 

Return A/N 



Theorem 11 The generic approximation algorithm QAA is a fully polynomial randomized approximation 
scheme (with additive error parameter) for computing p = Probk [ip] whenever ip is in the EPF fragment of 
LTL andpe]0,l[. 

One can obtain a randomized approximation of Prob[tp\ by iterating the approximation algorithm de- 
scribed above. Detection of time convergence for this algorithm is hard in general, but can be characterized 
for the important case of ergodic Markov chains. The logarithmic space complexity is an important feature 
for applications. 

Corollary 1 The fixed point algorithm defined by iterating the approximation algorithm QAA is a random- 
ized approximation scheme, whose space complexity is logspace, for the probability problem p = Prob[ip] 
whenever -0 is in the EPF fragment of LTL and p g]0, 1[. 

For ergodic Markov chains, the convergence rate of Probkii^] to Prob[tp] is in 0(fc™^^|A|'^) where A is the 
second eigenvalue of M and m its multiplicity. The randomized approximation algorithm described above 
is implemented in a distributed probabilistic model checker named APMC jHLP06| . Recently this tool has 
been extended to the verification of continuous time Markov chains. 

Randomized approximation scheme with multiplicative error. We use a generalization of the zero- 
one estimator theorem [KLM89 to estimate the expectation // of a random variable X distributed in the 
interval [0,1]. The generalized zero-one estimator theorem [DKLROO] proves that if Xi,X2, ■ ■ ■ ,Xiy are 
random variables independent and identically distributed according to X, S = X]j=i"^i' £ < 1, and N = 
4(e — 2).ln(|).p/(£./i)^, then S/N is an (e, (5)-approximation of ^, i.e.: 

Prob{fj.{l -e)< S/N < ^{1 + e)) > I - 5 

where p = max{a^,e(x) is a parameter used to optimize the number N of experiments and a^ denotes the 
variance of X. In jDKLROO] . an optimal approximation algorithm, running in three steps, is described: 

• using a stopping rule, the first step outputs an (e, (5)-approximation /t of /i after an expected number 
of experiments proportional to r//i where F = 4(e — 2). ln(|)/e^; 

• the second step uses the value of jl to set the number of experiments in order to produce an estimate 
p that is within a constant factor of p with probability at least (1 — (5); 

• the third step uses the values of jl and p to set the number of experiments and runs these experiments 
to produce an (e, (5)-approximation of /i. 

One obtains a randomized approximation scheme with multiplicative error by applying the optimal 
approximation algorithm OAA with input parameters e, 5 and the sample given by the function Random 
Path on a succinct representation of A^, the parameter k and the formula ip. 
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Theorem 12 The optimal approximation algorithm, OAA is a randomized approximation scheme (with 
multiplicative error) to compute p = Profefe [■(/;] whenever ip is in the EPF fragment of LTL and p g]0, 1[. 

We remark that the optimal approximation algorithm is not an FPRAS as the expected number of 
experiments r//i can be exponential for small values of /i. 

Corollary 2 The fixed point algorithm defined by iterating the optimal approximation algorithm OAA is 
a randomized approximation scheme for the probability problem p — Prob[ip\ whenever ip is in the EPF 
fragment of LTL and p g]0, 1[. 

5 Conclusion 

Model checking and testing are two areas with a similar goal: to verify that a system satisfies a property. 
They start with different hypothesis on the systems and develop many techniques with different notions of 
approximation, when an exact verification may be computationally too hard. 

We presented some of the well known notions of approximation with their logic and statistics backgrounds, 
which yield several techniques for model checking and testing. These methods guarantee the quality and the 
efficiency of the approximations. 

1. In bounded model checking, the approximation is on the length of the computation paths to witness 
possible errors, and the method is polynomial in the size of the model. 

2. In approximate model checking, we developped two approaches. In the first one, the approximation 
is on the density of errors and the Monte Carlo methods are polynomial in the size of the model. In 
the second one, the approximation is on the distance of the inputs and the complexity of the property 
testers is independent of the size of the model and only dependent on e. 

3. In approximate black box checking, learning techniques construct a model which can be compared with 
a property in exponential time. The previous approximate model checking technique guarantees that 
the model is e-close to the lUT after N samples, where N only depends on e. 

4. In approximate model-based testing, a coverage criterium is satisfied with high probability and the 
method is polynomial in the size of the representation. 

5. In approximate probabilistic model checking, the estimated probabilities of satisfying formulas are close 
to the real ones. The method is polynomial in the size of the given succinct representation. 

Some of these approximations can be combined for future research. For example, approximations used in 
black box checking and model-based testing can be merged, as learning methods infiuence the new possible 
tests. As another example, probabilistic model checking and approximate model checking can also be merged, 
as we may decide if a probabilistic system is close to satisfy a property. 
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